Add group_by option to the accessibility scanner#239
Conversation
| import type {Finding, ResolvedFiling, RepeatedFiling, FindingGroupIssue, Filing, IssueResponse} from './types.d.js' | ||
| import type { | ||
| Finding, | ||
| ResolvedFiling, | ||
| RepeatedFiling, | ||
| FindingGroupIssue, | ||
| Filing, | ||
| IssueResponse, | ||
| GroupBy, | ||
| } from './types.d.js' |
There was a problem hiding this comment.
yep! this must've hit the line length/import length limit
| "issue": {"id":1,"nodeId":"SXNzdWU6MQ==","url":"https://github.com/github/docs/issues/123","title":"Accessibility issue: 1"} | ||
| "issue": { | ||
| "id": 1, | ||
| "nodeId": "SXNzdWU6MQ==", | ||
| "url": "https://github.com/github/docs/issues/123", | ||
| "title": "Accessibility issue: 1" | ||
| } |
There was a problem hiding this comment.
Pull request overview
This PR adds a new group_by input to the accessibility scanner so users can consolidate many similar findings into fewer “umbrella” issues (by individual finding, by rule, or by rule+URL), reducing issue spam while keeping the default behavior unchanged.
Changes:
- Introduces a
group_byinput (defaultfinding) and wires it through the top-level action into thefileaction. - Updates filing/dedup logic to support grouping and renders grouped occurrences as a checklist in issue bodies (with occurrence count in the title).
- Adds/updates Vitest coverage for grouping behavior and invalid
group_byhandling.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents the new group_by input and its behavior. |
| action.yml | Declares group_by on the composite action and forwards it to the file step. |
| .github/actions/file/action.yml | Declares group_by input for the file action. |
| .github/actions/file/README.md | Documents group_by behavior for the file action. |
| .github/actions/file/src/types.d.ts | Adds the GroupBy union type. |
| .github/actions/file/src/updateFilingsWithNewFindings.ts | Makes dedup/grouping key computation group_by-aware and groups new filings by key. |
| .github/actions/file/src/index.ts | Reads/validates group_by, threads it into filing updates, and passes full findings arrays into open/reopen helpers. |
| .github/actions/file/src/openIssue.ts | Accepts findings arrays, updates title to include occurrence counts when grouped, and passes arrays to body generation. |
| .github/actions/file/src/reopenIssue.ts | Accepts findings arrays and regenerates bodies from the full set when reopening. |
| .github/actions/file/src/generateIssueBody.ts | Adds an Occurrences checklist section when there are multiple findings. |
| .github/actions/file/tests/updateFilingsWithNewFindings.test.ts | Adds tests covering finding vs rule vs rule+url grouping behavior. |
| .github/actions/file/tests/openIssue.test.ts | Updates tests for findings-array signature and grouped title behavior. |
| .github/actions/file/tests/reopenIssue.test.ts | Updates tests for findings-array signature. |
| .github/actions/file/tests/generateIssueBody.test.ts | Adds coverage for Occurrences rendering behavior. |
| .github/actions/file/tests/dryRun.test.ts | Adds dry-run coverage for grouping and invalid group_by fail-fast behavior. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 14/15 changed files
- Comments generated: 2
| for (const finding of filing.findings) { | ||
| findingKeys[getFindingKey(finding)] = getFilingKey(filing) | ||
| findingKeys[getFindingKey(finding, groupBy)] = getFilingKey(filing) | ||
| } |
| import {describe, it, expect} from 'vitest' | ||
| import {updateFilingsWithNewFindings} from '../src/updateFilingsWithNewFindings.ts' | ||
|
|
||
| const cc = (url: string, html: string) => ({ |
There was a problem hiding this comment.
can we give this an unabbreviated/real words name?
| let body: string | undefined | ||
| if (finding && repoWithOwner) { | ||
| body = generateIssueBody(finding, screenshotRepo ?? repoWithOwner) | ||
| if (findings && findings.length > 0 && repoWithOwner) { |
There was a problem hiding this comment.
| if (findings && findings.length > 0 && repoWithOwner) { | |
| if (findings?.length > 0 && repoWithOwner) { |
| : [] | ||
| const shouldOpenGroupedIssues = core.getBooleanInput('open_grouped_issues') | ||
| const groupByInput = core.getInput('group_by') || 'finding' | ||
| const validGroupByValues: GroupBy[] = ['finding', 'rule', 'rule+url'] |
There was a problem hiding this comment.
You can use something like this to ensure the type always remains in sync with its actual definition.
|
|
||
| export function generateIssueBody(finding: Finding, screenshotRepo: string): string { | ||
| export function generateIssueBody(findingOrFindings: Finding | Finding[], screenshotRepo: string): string { | ||
| const findings = Array.isArray(findingOrFindings) ? findingOrFindings : [findingOrFindings] |
There was a problem hiding this comment.
perhaps we can rename this to something which emphasizes this is duplicates of the same finding? this would improve readability
| if (findings.length > 1) { | ||
| const items = findings.map(f => `- [ ] ${f.html ? `\`${f.html}\` on ${f.url}` : f.url}`).join('\n') | ||
| occurrencesSection = ` | ||
| ## Occurrences (${findings.length}) |
There was a problem hiding this comment.
| ## Occurrences (${findings.length}) | |
| ## ${findings.length} Other Occurrences: |
| | `skip_copilot_assignment` | No | Whether to skip assigning filed issues to GitHub Copilot. Set to `true` if you don't have GitHub Copilot or prefer to handle issues manually | `true` | | ||
| | `include_screenshots` | No | Whether to capture screenshots of scanned pages and include links to them in filed issues. Screenshots are stored on the `gh-cache` branch of the repository running the workflow. Default: `false` | `true` | | ||
| | `open_grouped_issues` | No | Whether to create a tracking issue which groups filed issues together by violation type. Default: `false` | `true` | | ||
| | `group_by` | No | How to consolidate findings into issues: `finding` (default, one issue per individual violation), `rule` (one issue per rule, aggregating every occurrence across all scanned URLs), or `rule+url` (one issue per rule per scanned URL). Preferred over `open_grouped_issues` for consolidation. Default: `finding` | `rule` | |
There was a problem hiding this comment.
"default" should just be "finding"?
Tracking issue with context: https://github.com/github/accessibility/issues/10753
Based on feedback from
@joseinthearena: with many open scanner issues across a handful of rules (e.g. color-contrast ×11, heading-order ×6) there was no way to consolidate. Thegh-cachebranch maps each finding's fingerprint → issue URL 1:1, so re-routing N findings to a single "umbrella" issue meant editinggh-cacheJSON by hand — brittle, and a typo silently breaks dedup. This adds agroup_byinput that controls how findings are consolidated into issues:finding(default, current behavior — one issue per violation),rule(one issue per rule), orrule+url(one issue per rule per scanned URL). When grouping, each duplicate occurrence is appended to the single issue as a checklist item rather than spawning a new issue. Because the default isfinding, existing workflows behave exactly as before.Changes
action.yml(root) — Newgroup_byinput (defaultfinding); forwarded to thefilestep.github/actions/file/action.yml— Newgroup_byinput declared (defaultfinding)file/src/types.d.ts— NewGroupBytype ('finding' | 'rule' | 'rule+url')file/src/updateFilingsWithNewFindings.ts— The 1:1 dedup behavior lived entirely ingetFindingKey; it's now group-aware, computing a coarser key forrule/rule+urlso multiple findings collapse into a single filing carrying multiple findings (a shape the pipeline already supported). New findings in the same group are also de-duplicated into one new filingfile/src/index.ts— Reads and validatesgroup_by(fails fast viasetFailedon an invalid value), threads it intoupdateFilingsWithNewFindings, and passes each filing's full findings array to the open/reopen helpersfile/src/generateIssueBody.ts/openIssue.ts/reopenIssue.ts— When a filing groups multiple findings, each occurrence is rendered as a checklist item under an Occurrences section, and the issue title reflects the occurrence count; single-finding output is unchangedREADME.md—group_byinput documented in the inputs table and getting-started example.github/actions/file/README.md—group_byinput documentedTest Updates
file/tests/updateFilingsWithNewFindings.test.ts— assertsfindingyields one filing per violation (default),rulecollapses all occurrences of a rule into a single filing,rule+urlyields one filing per rule per URL, new occurrences attach to an existing cached filing instead of opening a new issue, and distinct rules stay separatefile/tests/openIssue.test.ts,generateIssueBody.test.ts, andreopenIssue.test.tsfor the findings-array signature and the Occurrences checklist renderingfile/tests/dryRun.test.tsto cover grouped dry-run behavior and invalid-group_byhandlingUsage
When
group_byis not provided, the scanner behaves exactly as before (one issue per finding).Notes
open_grouped_issuesoption as-is instead of removing it. It's similar but not the same (it still files every issue and adds a separate tracking issue, whilegroup_byfiles one combined issue). Removing it would be a breaking change, so I left it for a follow-up — happy to go either way.… (3 occurrences), since the findings can span multiple URLs.