Skip to content

ci: ensure migration benchmark runs#93

Merged
mrjf merged 1 commit into
mainfrom
codex/fix-migration-benchmark-actions
May 28, 2026
Merged

ci: ensure migration benchmark runs#93
mrjf merged 1 commit into
mainfrom
codex/fix-migration-benchmark-actions

Conversation

@mrjf

@mrjf mrjf commented May 28, 2026

Copy link
Copy Markdown
Contributor

TL;DR

This fixes the migration benchmark workflow so the Python-vs-Go CLI benchmark actually runs in Actions, uploads benchmark evidence, and posts benchmark results back to the PR for each head commit. It also removes the top-level PR path filter so the workflow is created on every PR update; Crane migration branches then force the migration checks to run on every commit.

What changed

  • Run the Python-vs-Go CLI benchmark immediately after building the Go CLI.
  • Keep the lightweight Python scaling guards, but stop running the broad stale Python-only benchmark suite in the migration workflow.
  • Fix the benchmark helper so read-only command benchmark working directories are created before subprocess execution.
  • Add a clear faster/slower result column to the benchmark markdown summary.
  • Trigger the migration workflow on every pull request update instead of relying on the workflow-level paths filter.
  • Force migration parity and benchmark jobs to run for crane/* PR branches, even when a commit only touches orchestration or generated metadata.
  • Post benchmark results as an idempotent PR comment keyed by the PR head SHA.

Validation

  • /opt/homebrew/bin/actionlint .github/workflows/migration-ci.yml
  • git diff --check HEAD~1 HEAD
  • PYTHONPYCACHEPREFIX=/private/tmp/apm-pycache python3 -m py_compile scripts/ci/migration_cli_benchmark.py
  • python3 scripts/ci/migration_cli_benchmark.py --python-bin /usr/bin/true --go-bin /usr/bin/true --json-out /private/tmp/migration-cli-benchmark-smoke.json --markdown-out /private/tmp/migration-cli-benchmark-smoke.md --repeats 1
  • GitHub Actions run 26557634231: Detect Migration Changes, Python-vs-Go Parity Gate, and Migration Benchmarks all passed.
  • PR benchmark comment posted for commit dbabdffd7801279bf4053a98026fa496f9505f9f.

How to test

  1. Push any commit to a crane/* migration PR branch.
  2. Confirm the Migration Parity and Benchmarks workflow is created for that commit.
  3. Confirm Detect Migration Changes, Python-vs-Go Parity Gate, and Migration Benchmarks all run.
  4. Confirm the PR receives a Migration Benchmark Results comment for that commit SHA.
  5. Confirm the migration-benchmark-evidence artifact includes JSON and markdown output.
@mrjf mrjf force-pushed the codex/fix-migration-benchmark-actions branch from 0b08b5b to 8faa590 Compare May 28, 2026 05:38
@mrjf mrjf force-pushed the codex/fix-migration-benchmark-actions branch from 8faa590 to dbabdff Compare May 28, 2026 05:59
@github-actions

Copy link
Copy Markdown
Contributor

Migration Benchmark Results

Migration CLI Benchmark

Max allowed Go/Python median ratio: 5.00

Command Python median Go median Go/Python Result Return codes
--help 0.4696s 0.0013s 0.00x 369.60x faster {'python': [0], 'go': [0]}
--version 0.4717s 0.0013s 0.00x 372.04x faster {'python': [0], 'go': [0]}
compile --help 0.4815s 0.0012s 0.00x 398.52x faster {'python': [0], 'go': [0]}
install --help 0.4818s 0.0013s 0.00x 372.27x faster {'python': [0], 'go': [0]}
pack --help 0.4798s 0.0013s 0.00x 379.79x faster {'python': [0], 'go': [0]}
audit --help 0.4688s 0.0013s 0.00x 367.88x faster {'python': [0], 'go': [0]}
init --yes 0.4973s 0.0013s 0.00x 382.47x faster {'python': [0], 'go': [0]}
@mrjf mrjf merged commit 7d2da66 into main May 28, 2026
13 checks passed
@mrjf mrjf deleted the codex/fix-migration-benchmark-actions branch May 28, 2026 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant