Add dry_run option to the accessibility scanner#232
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a dry_run mode to the accessibility-scanner GitHub Action so users can preview what a scan would do (open/reopen/close issues) via logs, without mutating issues, running Copilot fix/assignment, or writing updates to the gh-cache branch.
Changes:
- Introduces a new
dry_runinput in the rootaction.ymland forwards it into thefilesub-action; skips cache-write and other mutating steps when enabled. - Implements dry-run behavior in
.github/actions/file/src/index.tsto classify filings and log the actions that would be taken, while still writingfilings_file. - Adds dedicated dry-run test coverage and updates documentation (README + FAQ) to explain usage and behavior.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents the new dry_run input in the example and inputs table. |
| FAQ.md | Adds an FAQ entry describing how to preview scanner behavior using dry_run. |
| action.yml | Adds dry_run input, forwards it to the file step, and gates mutating/cache-write steps when enabled. |
| .github/actions/file/action.yml | Declares the dry_run input for the file sub-action. |
| .github/actions/file/src/index.ts | Implements dry-run logging/classification and skips Octokit issue mutations and grouped/tracking issue creation. |
| .github/actions/file/tests/dryRun.test.ts | Adds tests asserting no issue mutations happen in dry run and that expected logs/summary/output are produced. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 6/6 changed files
- Comments generated: 1
| dryRunCounts.reopen++ | ||
| core.info(`[dry run] Would REOPEN issue: ${filing.issue.url}`) | ||
| } | ||
| continue |
There was a problem hiding this comment.
To make the logic easier to follow, I'd suggest deleting the continue, then wrapping the not-dry-run code in an else block; this way, a reader will immediately see this is code which definitely doesn't execute for a dry run
There was a problem hiding this comment.
Refactored to if/else and dropped the continue, reads much clearer now
Co-authored-by: Joyce Zhu <joycezhu@github.com>
Tracking issue with context: https://github.com/github/accessibility/issues/10757
Based on feedback from
@joseinthearena: with many open scanner issues there was no way to preview what a scan would do without actually filing, closing, or reopening issues — and no way to do so without mutating thegh-cachebranch. This adds adry_runinput that runs a normal scan and logs the issues that would be filed, while making no changes: no issues are opened, closed, or reopened, no Copilot assignment, and the cache is not written. Because dry runs don't update the cache, the next real run behaves exactly as if the dry run never happened.Changes
action.yml(root) — Newdry_runinput (defaultfalse); forwarded to thefilestep; skips theFix,Save screenshots,Copy results to cache path, andSave cached resultssteps whendry_runistrue.github/actions/file/action.yml— Newdry_runinput declaredfile/src/index.ts— Whendry_runis set, each filing is classified via the existingisNewFiling/isRepeatedFiling/isResolvedFilingguards and the intended action is logged (earlycontinue, so no Octokit calls are made); grouped/tracking-issue creation is skipped; a summary count is logged;filings_fileis still written so the output contract is unchangedREADME.md—dry_runinput documented in the inputs table and getting-started exampleFAQ.md— New entry explaining how to preview a scan and noting that dry runs don't write the cacheTest Updates
file/tests/dryRun.test.ts— asserts no issues are opened/closed/reopened in dry run, the correct[dry run] Would …lines and summary are logged, andfilings_fileis still written; includes a regression guard that mutations still occur whendry_runisfalseUsage
When
dry_runis not provided, the scanner behaves exactly as before.Notes
file-action logic is covered by the new tests. Theaction.ymlstep gating was verified by inspection but not run end-to-end — flagging for confirmation in CI.v3tag re-pointed) cut by a maintainer.