Skip to content

Scope the root test script to tests/ so npm test runs from a clean install - #948

Merged
iamtoruk merged 1 commit into
getagentseal:mainfrom
therickfactr:fix/root-test-script-scope
Aug 10, 2026
Merged

Scope the root test script to tests/ so npm test runs from a clean install#948
iamtoruk merged 1 commit into
getagentseal:mainfrom
therickfactr:fix/root-test-script-scope

Conversation

@therickfactr

Copy link
Copy Markdown
Contributor

Problem

npm test does not work from a clean root install.

"test": "vitest" is unscoped, so vitest's default glob reaches the Electron app's specs under app/. Those carry their own app/vitest.config.ts and their own jsdom in app/package.json, installed into app/node_modules — which a root npm ci does not create. The run dies before it gets anywhere:

$ git clone … && npm ci && npm test
Error: Cannot find package 'jsdom' imported from …/node_modules/vitest/dist/chunks/index.CmSc2RE5.js
Serialized Error: { code: 'ERR_MODULE_NOT_FOUND' }
…
 Test Files  3 failed | 199 passed | 2 skipped (230)
     Errors  26 errors

This is already a known trap — .github/workflows/tests.yml documents it in a comment ("the root default glob picking them up is exactly what failed run #2 with ERR_MODULE_NOT_FOUND: jsdom") and works around it by spelling out a scoped vitest invocation in the workflow. But the workaround lives only in CI, so:

  • CONTRIBUTING.md tells a new contributor to run npm test, which fails on their first try.
  • RELEASING.md names npm test as the pre-release gate ("Before Every Release → Run the test suite"). That command currently errors out.

Change

Move the scoping CI already applies into package.json, and have CI call the scripts so the two cannot drift again:

script what it runs
test tests/, minus the parallelism-sensitive cache-refresh-lock suites
test:locks those three suites, serially (--poolOptions.forks.singleFork=true)
test:watch same scope as test, watch mode — preserves the old vitest behaviour

test (189 files) and test:locks (3) together cover all 192 files under tests/, so nothing is orphaned by the split.

CI behaviour is unchanged — this PR does not touch .github/workflows/. The test and test:locks strings are byte-identical to the invocations tests.yml already spells out (verified by string equality, not by eye), so if you want the two to stop drifting apart, those two run: lines can become npm test and npm run test:locks in a one-line follow-up. I left that edit out deliberately so this PR needs no workflow permissions — happy to add it if you would rather have it here.

One consequence worth calling out

Scoping the script changes what a trailing path argument means. vitest ORs positional filters, so npm test -- tests/providers/hermes.test.ts now resolves to vitest run tests --exclude "…" tests/providers/hermes.test.ts — which does not narrow to that file, it runs everything. Measured:

npx vitest list tests --exclude "tests/cache-refresh-lock*"                                 → 189 files
npx vitest list tests --exclude "tests/cache-refresh-lock*" tests/providers/codex.test.ts  → 189 files   (narrows nothing)
npx vitest list tests/providers/codex.test.ts                                              → 1 file

So every documented use of that idiom is rewritten to npx vitest run <path> — 13 lines across the four provider guides (opencode, hermes, codewhale, lingtai-tui) and docs/design/codeburn-mcp-plan.md. Without that, a contributor following docs/providers/hermes.md would expect a ten-second single-file run and get the full suite instead.

Docs refreshed alongside, because they had drifted:

  • CONTRIBUTING.md: "42 test files, 568 tests" → 189 of 192 files / 2,494 tests.
  • The provider test-gap list — CONTRIBUTING.md and docs/architecture.md both still name five providers without test files, but tests/providers/antigravity.test.ts and tests/providers/gemini.test.ts exist on main today. Corrected to the three that are genuinely missing (claude, goose, qwen). Pre-existing, but it sits in the sections this PR rewrites.
  • docs/architecture.md: per-directory counts (27/1/15 → 141/1/44, plus tests/sharing/), and the line "It does not run vitest in CI today", which stopped being true when tests.yml landed.
  • RELEASING.md: the pre-release gate now names npm run test:locks too, since CI treats those suites as reporting-only and a maintainer should eyeball them by hand before publishing.
  • CONTRIBUTING.md also gains a line making the lock-test convention explicit: a new cross-process-lock test must both match the tests/cache-refresh-lock-* prefix and be added to test:locks. Nothing recorded that before, and the split makes it load-bearing — a lock test named anything else runs under the full worker pool and flakes, while one that matches the prefix but is missing from test:locks silently never runs.

Verification

Run on a clean npm ci in a fresh worktree at 3536a1d:

  • Before: npx vitest run app/renderer/App.test.tsxERR_MODULE_NOT_FOUND: jsdom. Reproduced.
  • After: npm test → no jsdom error, 189 files / 2,494 tests collected.
  • Green: npx vitest run tests --exclude "tests/cache-refresh-lock*" --testTimeout=30000187 passed | 2 skipped (189 files), 2,489 passed | 5 skipped (2,494 tests), 0 failures.
  • npx tsc --noEmit → clean.

One caveat worth flagging rather than burying: at the default 5s testTimeout I could not get a green run on my machine — I saw 17, then 12, then 1 failure across runs, all timeouts, and the same flakiness on unmodified main at the same SHA (so it is not introduced here; upstream CI is green on 3536a1d). Raising only --testTimeout makes it uniformly green, which suggests the 5s default is tight for the suites that spawn CLI subprocesses on slower or loaded hardware. I have deliberately not touched the timeout in this PR — happy to open a separate one if you think it is worth doing.

npm run test:locks fails on my hardware, as #904 describes; it also fails there on unmodified main, and CI keeps it continue-on-error, so this PR does not change its status.

Notes

Related to #947 (release cadence) only in passing: the pre-release gate in RELEASING.md being a command that errors out seemed worth fixing regardless of when the next release lands.

…stall

vitest's default glob reached the Electron app's specs under app/, which carry their
own vitest config and their own jsdom in app/node_modules. From a root install that
fails with ERR_MODULE_NOT_FOUND: jsdom, so the command CONTRIBUTING documents and the
one RELEASING.md names as the pre-release gate both error out.

Move the scoping CI already applies into package.json: test runs tests/ minus the
parallelism-sensitive cache-refresh-lock suites, test:locks runs those three serially,
test:watch keeps watch mode at the same scope. The first two are byte-identical to the
invocations .github/workflows/tests.yml spells out, so the workflow can be pointed at
the scripts to stop the two drifting apart again; that edit is left out of this PR so it
needs no workflow permissions. test plus test:locks together still cover all 192 files
under tests/.

Scoping the script changes what a trailing path argument means: vitest ORs positional
filters, so 'npm test -- tests/providers/hermes.test.ts' would no longer narrow to that
file, it would run the whole suite. Rewrite those to 'npx vitest run <path>' everywhere
they appear - four provider guides and the MCP design plan, thirteen lines in all.

Also refresh the stale test docs: 42 files/568 tests (now 192 under tests/), the
per-directory counts, the line claiming vitest does not run in CI which stopped being
true when tests.yml landed, and the provider test-gap list, which still named
antigravity and gemini after both gained test files.

Record the cache-refresh-lock naming convention in CONTRIBUTING, since the split makes
it load-bearing: a lock test that misses the prefix runs under the full worker pool and
flakes, and one that matches it but is absent from test:locks never runs at all.
@iamtoruk
iamtoruk merged commit a6446d9 into getagentseal:main Aug 10, 2026
5 checks passed
pull Bot pushed a commit to TheTechOddBug/codeburn that referenced this pull request Aug 11, 2026
cli-json-daily, spend-flow and cli-emitters spawn the real CLI per test
and blow the 5s default under full parallel suite load while passing in
isolation - the flake set getagentseal#948 documented on unmodified main, observed
again locally (cli-json-daily) and in CI (cli-emitters on a green PR).
Same file-level remedy the CLI menubar suite already uses; the default
stays 5s for everything else.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants