Repository navigation
build: cut the gates job's run time - #1086
Conversation
The gates job reran the whole engine suite optimized, about 7,600 tests the Build job had already run, on top of the ignored tests it exists for. scripts/gates.sh with no arguments now runs the ignored tests only. It still builds with ext_data on and runs that feature's own tests by name: they are not ignored, and no other job compiles them, so without that they would run nowhere. A filter still runs what it names, ignored or not.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
ReviewNo blocking issues found. The script correctly:
Verified that only Overall correctness verdictCorrect. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1086 +/- ##
=======================================
Coverage 92.90% 92.90%
=======================================
Files 313 313
Lines 194144 194144
=======================================
+ Hits 180363 180369 +6
+ Misses 13781 13775 -6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
On a CI runner the gates job's run step is one test: the property test that sends every tool through generated models runs its 200 cases one after another on one thread, about eleven minutes, and for the last six of them it is the only test still running. Forty cases is about two minutes. Each run draws new shapes and replays every saved failing seed, so coverage accumulates across runs.
ReviewReviewed the trim to What I verified:
Overall correctness verdict: correct. Existing behavior is preserved for the filter and |
Why
The gates job is the longest job in CI: about 20 minutes on main (5.5 to 8 to compile, 10 to 13 to run on the last four runs), past the macOS jobs at about 15.
What
scripts/gates.shwith no arguments runs the ignored tests only (47 today), not the whole engine suite again. It still builds with theext_datafeature (the Excel data provider) and runs that feature's own tests by name (data_provider::, 43 tests, under a second): they are not ignored, and no other job compiles them.scripts/gates.sh <filter>is unchanged.Evidence
Measured on this PR's own gates job:
Running only the ignored tests saved nothing by itself: the already-run tests cost seconds once optimized. The property test was the cost. After the change the unit gates finish in 4.6 minutes and the integration gates in 1.7.
What this does not establish
🤖 Generated with Claude Code
https://claude.ai/code/session_01LYR7N9WZwQuFnZFW8JDUgx