Skip to content

Optimize /api/runs/list job loading#3986

Merged
peterschmidt85 merged 4 commits into
masterfrom
fix/runs-list-performance
Jun 25, 2026
Merged

Optimize /api/runs/list job loading#3986
peterschmidt85 merged 4 commits into
masterfrom
fix/runs-list-performance

Conversation

@peterschmidt85

@peterschmidt85 peterschmidt85 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Fixes #3983

Problem

/api/runs/list powers both the Runs UI and dstack ps. UI/CLI list views normally need only the latest job submission, but the server still eager-loaded the full RunModel.jobs history and trimmed it later during response serialization.

On runs with many retries/submissions, one page could materialize tens of thousands of JobModel rows. Admin/all-project requests also had extra overhead because the service loaded accessible projects into Python and filtered runs with a large project_id IN (...) list.

Fix

  • Stop eager-loading RunModel.jobs in the runs-list path.
  • After selecting the requested page of runs, load only the job rows needed for the response.
  • When job_submissions_limit is set, load the latest N submissions per (run_id, replica_num, job_num) with a SQL window query.
  • Also load the latest terminated submission with a termination reason per logical job, so status messages such as retrying stay correct without loading full history.
  • Replace Python-built project ID lists with direct project lookup/subqueries for single-project, admin all-project, and member all-project paths.
  • Ensure lightweight UI/CLI list callers pass job_submissions_limit=1.
@peterschmidt85 peterschmidt85 requested a review from r4victor June 24, 2026 20:23
@peterschmidt85 peterschmidt85 force-pushed the fix/runs-list-performance branch from 2bb1a37 to 95fb10f Compare June 25, 2026 07:49
@peterschmidt85 peterschmidt85 requested a review from jvstme June 25, 2026 10:07
Comment thread src/dstack/_internal/server/services/runs/__init__.py Outdated
Comment thread src/dstack/_internal/server/services/runs/__init__.py Outdated
@peterschmidt85 peterschmidt85 merged commit 412a58b into master Jun 25, 2026
22 checks passed
@peterschmidt85 peterschmidt85 deleted the fix/runs-list-performance branch June 25, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants