Skip to content

fix(aws-lambda): fix flake in aws lambda tests - #6937

Merged
pabloDeputter merged 5 commits into
masterfrom
pablo/fix-flake-aws-lambda-test
Jul 30, 2026
Merged

fix(aws-lambda): fix flake in aws lambda tests#6937
pabloDeputter merged 5 commits into
masterfrom
pablo/fix-flake-aws-lambda-test

Conversation

@pabloDeputter

@pabloDeputter pabloDeputter commented Jul 29, 2026

Copy link
Copy Markdown
Member

Description

  • Catch subprocess.TimeoutExpired during SAM teardown and force-kill, so slow shutdown cannot fail the entire test suite.
  • Added temporary flake-check workflow that runs each test aws-lambda test 5 times to verify this removes the flake (workflow is removed in the last commit; was just to verify that it works). [https://github.com/getsentry/sentry-python/actions/runs/30539137588/job/90862136512?pr=6937](last successful run)
  • Switched SAM --warm-containers startup from EAGER to LAZY so Docker containers start on first invocation instead of all 21 at SAM startup. Verified locally by running TESTPATH=tests/integrations/aws_lambda/test_aws_lambda.py uv run tox -e py3.8-aws_lambda both for EAGER & LAZY 10x independently:
EAGER:
=================================================
Metric              |   Mean (s) |  Std Dev (s)
=================================================
Setup time          |      43.69 |         3.99
First call time     |       1.92 |         0.22
Max Teardown time   |       7.49 |         0.27
Total run time      |     108.53 |         4.56
=================================================
LAZY:
=================================================
Metric              |   Mean (s) |  Std Dev (s)
=================================================
Setup time          |      33.32 |         1.82
First call time     |       6.32 |         0.63
Max Teardown time   |       6.90 |         0.18
Total run time      |     102.06 |         3.60
=================================================

Issues

Resolves: #6911 & PY-2632

Reminders

…ture & added temporary workflow that runs aws_lambda tests in parallel

Refs: #6911
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

99295 passed | ⏭️ 6781 skipped | Total: 106076 | Pass Rate: 93.61% | Execution Time: 351m 12s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +420
Passed Tests 📈 +417
Failed Tests 📉 -1
Skipped Tests 📈 +4

All tests are passing successfully.

✅ Patch coverage is 100.00%. Project has 2498 uncovered lines.
✅ Project coverage is 89.87%. Comparing base (base) to head (head).

Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    89.87%    89.87%        —%
==========================================
  Files          193       193         —
  Lines        24656     24656         —
  Branches      8786      8786         —
==========================================
+ Hits         22158     22158         —
- Misses        2498      2498         —
- Partials      1400      1400         —

Generated by Codecov Action

- each container now will be initiated on first invocation

Refs: #6937
@pabloDeputter pabloDeputter changed the title fix(aws-lambda): still drafting Jul 30, 2026
Drop the temporary flake-check workflow, restore a 10s shutdown budget (local teardown averages ~7s), and escalate to kill on timeout so an uncaught TimeoutExpired cannot fail the suite.
@pabloDeputter
pabloDeputter marked this pull request as ready for review July 30, 2026 12:14
@pabloDeputter
pabloDeputter requested a review from a team as a code owner July 30, 2026 12:14

@alexander-alderman-webb alexander-alderman-webb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@sentrivana sentrivana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@pabloDeputter
pabloDeputter merged commit cbfb30e into master Jul 30, 2026
271 of 273 checks passed
@pabloDeputter
pabloDeputter deleted the pablo/fix-flake-aws-lambda-test branch July 30, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants