Skip to content

perf(ci): (more) sharding of browser and runtime integration tests - #26163

Merged
nicohrubec merged 1 commit into
feat/reduce-playwright-system-depsfrom
feat/benchmark-ci-sharding
Nov 6, 2026
Merged

nicohrubec merged 1 commit into
feat/reduce-playwright-system-depsfrom
feat/benchmark-ci-sharding

Conversation

@nicohrubec

@nicohrubec nicohrubec commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Why: Shard or increase sharding of some of the browser and runtime integration tests that became bottlenecks in recent runs. Wall time for full CI runs is expected to only slightly improve, but this should greatly benefit completion time for more targeted runs that only kick off integration tests at the cost of higher overall CI minutes. Chromium CDN variants and WebKit use two shards; required Node versions and Cloudflare use four.

Slowest test-execution step in each group, excluding setup and queueing:

Tests Baseline More shards Improvement
Chromium CDN bundles 2m30 1m53 25%
WebKit 2m24 1m36 33%
Node 20 4m48 3m56 18%
Node 22 4m38 3m59 14%
Node 24 3m55 3m15 17%
Node 26 3m39 3m08 14%
Cloudflare 4m09 2m04 50%

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Compared: 202d0d7 (PR test merge using feat/reduce-playwright-system-deps at 291ac82)

Against: f2ee787 (feat/reduce-playwright-system-deps) · base run #65001

⚠️ Baseline mismatch: 19 feat/reduce-playwright-system-deps commit(s) in the PR build are missing from the baseline. Their size changes also affect this comparison. Compare commits. Re-run once feat/reduce-playwright-system-deps's baseline build for 291ac82 finishes.

Path Size % Change Change
@sentry/browser 30.4 kB - -
@sentry/browser - with treeshaking flags 28.54 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 28.43 kB - -
@sentry/browser (incl. Tracing) 52.72 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.72 kB - -
@sentry/browser (incl. Tracing, Profiling) 55.69 kB - -
@sentry/browser (incl. Tracing, Replay) 93.02 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 81.61 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 97.76 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 110.75 kB - -
@sentry/browser (incl. Feedback) 47.98 kB - -
@sentry/browser (incl. sendFeedback) 35.5 kB - -
@sentry/browser (incl. FeedbackAsync) 40.61 kB - -
@sentry/browser (incl. Metrics) 31.44 kB - -
@sentry/browser (incl. Logs) 31.73 kB - -
@sentry/browser (incl. Metrics & Logs) 32.45 kB - -
@sentry/react 32.29 kB - -
@sentry/react (incl. Tracing) 55.09 kB - -
@sentry/vue 38.67 kB - -
@sentry/vue (incl. Tracing) 55.73 kB - -
@sentry/svelte 30.43 kB - -
@sentry/remix (Remix 3 client bundle) 57.8 kB - -
CDN Bundle 32.19 kB - -
CDN Bundle (incl. Tracing) 53.24 kB - -
CDN Bundle (incl. Logs, Metrics) 34.47 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 55.19 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 75.88 kB - -
CDN Bundle (incl. Tracing, Replay) 91.46 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 93.39 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 97.56 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 99.56 kB - -
CDN Bundle - uncompressed 94.67 kB - -
CDN Bundle (incl. Tracing) - uncompressed 157.63 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 101.14 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 163.37 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 232.71 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 279.32 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 285.05 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 293.03 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 298.74 kB - -
@sentry/nextjs (client) 58.04 kB - -
@sentry/sveltekit (client) 53.23 kB - -
@sentry/core/server 44.25 kB - -
@sentry/core/browser 13.93 kB - -
@sentry/node 155.71 kB +0.02% +18 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 82.26 kB - -
@sentry/node - without tracing 96.89 kB +0.02% +17 B 🔺
@sentry/node - without channel injection 136.04 kB +0.02% +22 B 🔺
@sentry/aws-serverless 105.11 kB +0.02% +21 B 🔺
@sentry/cloudflare (withSentry) - minified 200.67 kB - -
@sentry/cloudflare (withSentry) 499.05 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 221.22 kB - -

View base workflow run

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Optional E2E tests

Some optional checks were not successful. They do not block the merge. 5 not successful, 50 successful. See the run.

Check Status
❌ CI: Optional E2E Tests / E2E cloudflare-pi-durable (latest) Test (optional) Failing after 31s Details
❌ CI: Optional E2E Tests / E2E node-eve (latest) Test (optional) Failing after 1m Details
❌ CI: Optional E2E Tests / E2E node-flue (latest) Test (optional) Failing after 27s Details
❌ CI: Optional E2E Tests / E2E node-eve (orchestrion) Test (optional) Failing after 2m Details
❌ CI: Optional E2E Tests / E2E node-eve Test (optional) Failing after 2m Details
Successful checks
Check Status
✅ CI: Optional E2E Tests / E2E cloudflare-flue-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E debug-id-sourcemaps Test (optional) Successful in 41s Details
✅ CI: Optional E2E Tests / E2E astro-7-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E deno (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries Test (optional) Successful in 55s Details
✅ CI: Optional E2E Tests / E2E cloudflare-agent Test (optional) Successful in 40s Details
✅ CI: Optional E2E Tests / E2E astro-7 Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v2 Test (optional) Successful in 31s Details
✅ CI: Optional E2E Tests / E2E cloudflare-mastra Test (optional) Successful in 51s Details
✅ CI: Optional E2E Tests / E2E cloudflare-think Test (optional) Successful in 53s Details
✅ CI: Optional E2E Tests / E2E nextjs-16-tunnel (cloudflare) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-think (ai v6) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (cloudflare, latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-orpc Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (latest) Test (optional) Successful in 52s Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable Test (optional) Successful in 48s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions Test (optional) Successful in 28s Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (cloudflare) Test (optional) Successful in 43s Details
✅ CI: Optional E2E Tests / E2E cloudflare-workers-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext, bun) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v1 Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 Test (optional) Successful in 42s Details
✅ CI: Optional E2E Tests / E2E nuxt-5 (tunnel-dev) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E supabase-nextjs Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (turbo) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-flue Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-pi-durable Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E node-mastra (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-router-7-framework-instrumentation Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (client source maps kept) Test (optional) Successful in 40s Details
✅ CI: Optional E2E Tests / E2E node-pi-durable (latest) Test (optional) Successful in 36s Details
✅ CI: Optional E2E Tests / E2E nuxt-5 Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (deno) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (latest) Test (optional) Successful in 48s Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 (tracer provider) Test (optional) Successful in 41s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-sourcemaps Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E node-mastra Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (latest) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E vue-3 (no Options API) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions (latest) Test (optional) Successful in 31s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E solidstart-2 Test (optional) Successful in 58s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (static / nuxt generate) Test (optional) Successful in 34s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-cloudflare Test (optional) Successful in 39s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext) Test (optional) Successful in 1m Details

@nicohrubec nicohrubec changed the title ci: benchmark additional integration test sharding ci: shard browser and runtime integration tests to reduce CI latency Nov 6, 2026
@nicohrubec nicohrubec changed the title ci: shard browser and runtime integration tests to reduce CI latency perf(ci): shard browser and runtime integration tests to reduce wall time Nov 6, 2026
Co-Authored-By: OpenAI Codex <codex@openai.com>
@nicohrubec
nicohrubec changed the base branch from feat/use-pnpm-pack to feat/reduce-playwright-system-deps November 6, 2026 09:46
@nicohrubec
nicohrubec force-pushed the feat/benchmark-ci-sharding branch from 9b9f48f to e729160 Compare November 6, 2026 09:46
@nicohrubec
nicohrubec added this pull request to stack #26166 November 6, 2026 09:48
@nicohrubec
nicohrubec marked this pull request as ready for review November 6, 2026 09:49
@nicohrubec
nicohrubec requested review from a team as code owners November 6, 2026 09:49
@nicohrubec
nicohrubec requested review from andreiborza, chargome, isaacs, logaretm, msonnb and mydea and removed request for a team November 6, 2026 09:49
@nicohrubec nicohrubec changed the title perf(ci): shard browser and runtime integration tests to reduce wall time perf(ci): (more) sharding of browser and runtime integration tests Nov 6, 2026
@nicohrubec
nicohrubec removed this pull request from stack #26166 November 6, 2026 10:30
@nicohrubec
nicohrubec added this pull request to stack #26168 November 6, 2026 10:30
@nicohrubec
nicohrubec merged commit 18c69eb into develop Nov 6, 2026
435 of 440 checks passed
@nicohrubec
nicohrubec deleted the feat/benchmark-ci-sharding branch November 6, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants