fix(builders): restore discovery WeakMap cache for dev rebuilds#1699
fix(builders): restore discovery WeakMap cache for dev rebuilds#1699TooTallNate wants to merge 1 commit intomainfrom
Conversation
Move `workflow/runtime` resolution from `createStepsBundle` into `discoverEntries` so callers pass the original `inputFiles` array reference. This restores the WeakMap cache hit when `createWorkflowsBundle` and `createStepsBundle` are called with the same inputFiles, avoiding a redundant second esbuild discovery pass per build that was causing E2E Local Dev Tests to flake on CI.
🦋 Changeset detectedLatest commit: bf45928 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (74 failed)mongodb (7 failed):
redis (7 failed):
turso (60 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
There was a problem hiding this comment.
Pull request overview
This PR fixes a performance regression in @workflow/builders where discoverEntries()’s WeakMap cache stopped hitting during dev rebuilds, causing duplicate esbuild “discovery” passes and flaky CI timeouts.
Changes:
- Moves
workflow/runtimeresolution intodiscoverEntries()so callers can keep passing the originalinputFilesarray reference (restoring WeakMap cache hits). - Updates
createStepsBundle()to calldiscoverEntries(inputFiles, ...)directly (removing the “always new array”discoveryInputslogic). - Adds a changeset to ship the fix as a patch release.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| packages/builders/src/base-builder.ts | Resolves workflow/runtime inside discoverEntries() and uses internal entryPoints while caching by the original inputs reference. |
| .changeset/fix-discovery-weakmap-cache.md | Patch changeset documenting the dev rebuild performance fix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
Fixes flaky E2E Local Dev Tests that started failing after #1669.
workflow/runtimeresolution fromcreateStepsBundle()intodiscoverEntries()so callers pass the originalinputFilesarray referenceWeakMap<string[], DiscoveredEntries>cache hit whencreateWorkflowsBundle()andcreateStepsBundle()are called sequentially with the sameinputFilesRoot Cause
PR #1669 added
workflow/runtimeto the discovery inputs insidecreateStepsBundle:Since
resolvedWorkflowRuntimeis almost always truthy, this always created a new array, causing the WeakMap cache (keyed on array reference) to never hit. Eachbuild()call ran two full esbuild discovery passes instead of one. Combined with the broader/.*/onResolve filter (also from #1669), this roughly doubled an already-slower build, pushing total rebuild time past the 50-second polling timeout on CI.Fix
The
workflow/runtimeresolution now lives insidediscoverEntries()itself. The expanded entry points list is used internally for the esbuild build, but the originalinputsarray is preserved as the WeakMap key. BothcreateWorkflowsBundleandcreateStepsBundlepass the sameinputFilesreference and share the cache.