Skip to content

Deflake the issue-1363 tests: wait for lifespan startup instead of sleeping#2879

Merged
maxisbey merged 1 commit into
mainfrom
maxisbey/deflake-1363-startup-handshake
Jun 15, 2026
Merged

Deflake the issue-1363 tests: wait for lifespan startup instead of sleeping#2879
maxisbey merged 1 commit into
mainfrom
maxisbey/deflake-1363-startup-handshake

Conversation

@maxisbey

Copy link
Copy Markdown
Contributor

The three tests in tests/issues/test_1363_race_condition_streamable_http.py synchronize with their server thread via a fixed await anyio.sleep(0.1). On a loaded CI runner the thread sometimes isn't ready within that window, and the test fails with RuntimeError: Task group is not initialized. Make sure to use run(). This replaces the fixed sleep with a readiness handshake.

Motivation and Context

These tests (added in #1384) start a ServerThread that spins up its own event loop, enters the Starlette lifespan, and only then sets the session manager's task group. The test waits a fixed 0.1s and then sends requests from its own event loop via httpx.ASGITransport. When the thread loses the scheduling race — easy under pytest -n auto plus coverage on a 4-vCPU runner — the first request reaches handle_request() while _task_group is still None and the test fails.

This has flaked 6 times since 2026-05-27, on both Ubuntu and Windows, across unrelated PRs — e.g. 3.14/locked/windows and 3.14/locked/ubuntu. Because the test job is continue-on-error, these never turn a run red, so they're easy to miss.

The fix: the server thread sets a threading.Event once lifespan startup has completed (at that point the task group is guaranteed to exist), and the tests wait on it (bounded at 5s, via anyio.to_thread.run_sync) instead of sleeping. The trailing anyio.sleep(0.2) that gives the original #1363 race its detection window is deliberately left untouched — the tests still exercise exactly the same request paths and log checks.

How Has This Been Tested?

  • The three tests pass, and pass 150/150 with --flake-finder --flake-runs=50 -n 4.
  • Failure-mode A/B with an artificial 0.3s delay injected at the top of ServerThread.run(): the old tests reproduce the exact CI error, the fixed tests pass. Repeated on a windows-latest runner (Python 3.14): unpatched + delay fails all three tests with the same RuntimeError; patched + delay passes; patched 50× per test under -n 4 passes 150/150.
  • ./scripts/test passes at 100% coverage; ruff/pyright clean.

Breaking Changes

None — test-only change.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

All three tests in the file shared the same fixed-sleep pattern; only test_race_condition_invalid_accept_headers happened to be the one observed failing in CI, but the delayed-thread experiment shows the other two fail the same way, so all three call sites are converted.

AI Disclaimer

…eeping

The three tests in test_1363_race_condition_streamable_http.py waited a
fixed 0.1s for the ServerThread to start the app lifespan before sending
requests. On a loaded CI runner the thread is sometimes not ready in time,
so the first request reaches handle_request() before the session manager's
task group exists and the test fails with "RuntimeError: Task group is not
initialized" (seen intermittently on both Ubuntu and Windows jobs).

Replace the fixed sleep with a threading.Event that the server thread sets
once lifespan startup has completed; the tests wait for it (bounded at 5s)
before sending the first request.
@maxisbey maxisbey marked this pull request as ready for review June 15, 2026 15:48
@maxisbey maxisbey enabled auto-merge (squash) June 15, 2026 15:49
@maxisbey maxisbey merged commit ddb7b78 into main Jun 15, 2026
30 checks passed
@maxisbey maxisbey deleted the maxisbey/deflake-1363-startup-handshake branch June 15, 2026 15:51
@github-actions

Copy link
Copy Markdown
Contributor

This pull request is included in pre-release v2.0.0a2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants