Skip to content

fix(aws-lambda): fix flake in aws lambda tests - #6937

Merged
pabloDeputter merged 5 commits into
masterfrom
pablo/fix-flake-aws-lambda-test
Jul 30, 2026
Merged

fix(aws-lambda): fix flake in aws lambda tests#6937
pabloDeputter merged 5 commits into
masterfrom
pablo/fix-flake-aws-lambda-test

Conversation

@pabloDeputter

@pabloDeputter pabloDeputter commented Jul 29, 2026

Copy link
Copy Markdown
Member

Description

  • Catch subprocess.TimeoutExpired during SAM teardown and force-kill, so slow shutdown cannot fail the entire test suite.
  • Added temporary flake-check workflow that runs each test aws-lambda test 5 times to verify this removes the flake (workflow is removed in the last commit; was just to verify that it works). [https://github.com/getsentry/sentry-python/actions/runs/30539137588/job/90862136512?pr=6937](last successful run)
  • Switched SAM --warm-containers startup from EAGER to LAZY so Docker containers start on first invocation instead of all 21 at SAM startup. Verified locally by running TESTPATH=tests/integrations/aws_lambda/test_aws_lambda.py uv run tox -e py3.8-aws_lambda both for EAGER & LAZY 10x independently:
EAGER:
=================================================
Metric              |   Mean (s) |  Std Dev (s)
=================================================
Setup time          |      43.69 |         3.99
First call time     |       1.92 |         0.22
Max Teardown time   |       7.49 |         0.27
Total run time      |     108.53 |         4.56
=================================================
LAZY:
=================================================
Metric              |   Mean (s) |  Std Dev (s)
=================================================
Setup time          |      33.32 |         1.82
First call time     |       6.32 |         0.63
Max Teardown time   |       6.90 |         0.18
Total run time      |     102.06 |         3.60
=================================================

Issues

Resolves: #6911 & PY-2632

Reminders

…ture & added temporary workflow that runs aws_lambda tests in parallel

Refs: #6911
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

99295 passed | ⏭️ 6781 skipped | Total: 106076 | Pass Rate: 93.61% | Execution Time: 351m 12s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +420
Passed Tests 📈 +417
Failed Tests 📉 -1
Skipped Tests 📈 +4

All tests are passing successfully.

✅ Patch coverage is 100.00%. Project has 2498 uncovered lines.
✅ Project coverage is 89.87%. Comparing base (base) to head (head).

Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    89.87%    89.87%        —%
==========================================
  Files          193       193         —
  Lines        24656     24656         —
  Branches      8786      8786         —
==========================================
+ Hits         22158     22158         —
- Misses        2498      2498         —
- Partials      1400      1400         —

Generated by Codecov Action

- each container now will be initiated on first invocation

Refs: #6937
@pabloDeputter pabloDeputter changed the title fix(aws-lambda): still drafting fix(aws-lambda): fix flake in aws lambda tests Jul 30, 2026
Drop the temporary flake-check workflow, restore a 10s shutdown budget (local teardown averages ~7s), and escalate to kill on timeout so an uncaught TimeoutExpired cannot fail the suite.
@pabloDeputter
pabloDeputter marked this pull request as ready for review July 30, 2026 12:14
@pabloDeputter
pabloDeputter requested a review from a team as a code owner July 30, 2026 12:14

@alexander-alderman-webb alexander-alderman-webb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@sentrivana sentrivana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@pabloDeputter
pabloDeputter merged commit cbfb30e into master Jul 30, 2026
271 of 273 checks passed
@pabloDeputter
pabloDeputter deleted the pablo/fix-flake-aws-lambda-test branch July 30, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flake AWS lambda test

3 participants