fix(cli)!: fail test and pw-test when no checks match, and unify CI detection

[hlenke]

Affected Components

• CLI
• Create CLI
• Test
• Docs
• Examples
• Other

What

Three small fixes to how the CLI ends a run and how it decides it is in CI.

1. "No checks found" passed silently. checkly test and checkly pw-test printed "Unable to find checks to run" and then exited 0. A typo in a file filter, or a Playwright config that matches nothing, turned a CI job green while testing nothing. checkly trigger and checkly checks run already fail in that case.

2. Two raw process.exit calls. Both stopped the process in the middle of a command:

• Cancelling a checkly init prompt called process.exit(0) from inside the prompt library.
• A second Ctrl+C during a run called process.exit(1), skipping the signal handlers registered before the run.

3. Two ways to detect CI that disagreed. The default reporter used ci-info's isCI, while everything else (the x-checkly-source header, confirmations, prompts, skills) used detectCliMode(). For an agent inside GitHub Actions, or with CHECKLY_CLI_MODE set, the two gave different answers.

Fix

1. test and pw-test take --[no-]fail-on-no-matching, on by default and identical to the flag on trigger and checks run. When nothing matches they set exit code 1.
2. • makeOnCancel throws Errors.ExitError(0), which oclif turns into a clean exit 0 with output flushed.
   • The second Ctrl+C puts back the SIGINT listeners from before the run and re-raises the signal, so those listeners (e.g. when-exit) get to clean up.
3. detectCliMode() is the single source of truth. detectOperator() now also checks ci-info's isCI, so CI vendors that set no CI variable (Jenkins, TeamCity, CodeBuild) are still detected, and CI=false now means "not CI". The default reporter is list in interactive mode and ci otherwise. trigger.ts used its own copy of prepareReportersTypes; it now uses the shared one.

Breaking / behaviour changes

• checkly test and checkly pw-test exit 1 when no checks match. Pass --no-fail-on-no-matching for the old behaviour. This is why the title has !.
• checkly test --list with no matches also exits 1, because the no-checks branch runs before --list.
• Jenkins, TeamCity and CodeBuild now run in ci mode instead of interactive. For example, checkly deploy without --force there now prints the JSON preview and exits 2, where before it showed a prompt nobody could answer.
• Agents get the ci reporter by default instead of list.
• A double Ctrl+C exits 130 (SIGINT) on macOS/Linux instead of 1. On Windows, re-raising SIGINT terminates the process immediately, as before.

Notes for the Reviewer

• ci-info reads the environment once at import, so any spec that calls the real detectOperator() must mock it. Without the mock, three api.spec.ts tests fail only in CI. cli-mode.spec.ts and api.spec.ts now mock it, and the full unit suite passes both normally and with CI=true GITHUB_ACTIONS=true.
• The e2e harness forces CHECKLY_CLI_MODE=interactive, so in GitHub Actions e2e now gets the list reporter, the same one local e2e runs always used. A new e2e case runs --reporter ci explicitly so that reporter keeps end-to-end coverage.
• Open question: should test --list with no matches stay at exit 0, since it is a query rather than a run?

Testing

• Unit: cli-mode.spec.ts covers a CI vendor with no CI variable and CI=false. test-helper.spec.ts covers the default reporter in ci/agent mode. abstract-check-runner.spec.ts asserts the second SIGINT is re-raised rather than process.exit. A new prompts-helpers.spec.ts covers the cancel exit.
• e2e (test.spec.ts): no matching checks now expects exit 1, --no-fail-on-no-matching expects exit 0, plus a new --reporter ci case. Not run locally (needs the e2e account); this draft is open to run them.
• pnpm lint, tsc --noEmit and the full unit suite (227 files) pass locally and under simulated CI env.

No new commands, so no backend/API dependency. Help output gains --[no-]fail-on-no-matching on test and pw-test.

🤖 Generated with Claude Code