Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
d63e5db
M0: hidden `quarto dev-call axe` skeleton command
cwickham Aug 20, 2026
f320513
M1: scan an already-rendered site end-to-end
cwickham Aug 20, 2026
84f1b6e
Scan the whole document: drop the axe context excludes
cwickham Aug 20, 2026
1dc920e
M2: aggregate, report and the two contracts
cwickham Aug 20, 2026
3bc8d9b
Version the signature scheme, and pin the normalization with tests
cwickham Aug 20, 2026
10e7628
Let Zod's .strict() do the baseline error reporting
cwickham Aug 20, 2026
0f79345
M3: fixture site, unit tests and a browser smoke test
cwickham Aug 20, 2026
64d2612
M4: survive the bundle, and stop importing a browser module to do it
cwickham Aug 20, 2026
44494ec
M4a: select the author's themes, not just the user's OS preference
cwickham Aug 20, 2026
8162c39
Fix: light-only colour-scheme links are not a dark theme
cwickham Aug 20, 2026
e7f8563
Split the axe fixture site into one site per concern
cwickham Aug 20, 2026
9f3545c
M4b: per-page mode discovery replaces the runtime theme probe
cwickham Aug 20, 2026
40f59d7
Never discover the scanner's own output or vendored libs as pages
cwickham Aug 25, 2026
84127d4
B1: a rejected CDP send fails its cell closed, not the whole scan
cwickham Aug 25, 2026
bee408a
Fail closed when the page redirects away from the served site
cwickham Aug 25, 2026
b7719c7
Wait for readiness instead of a fixed 500ms settle per cell
cwickham Aug 25, 2026
d7f0df7
Add --exclude: site-relative globs to skip, after --pages
cwickham Aug 25, 2026
5739f5f
Skip redirect stubs at discovery, and harden the scan per review
cwickham Aug 25, 2026
ce9e11b
Ship report.md and a generated README; retire the HTML report
cwickham Aug 26, 2026
cc31326
report.md: link table rows to occurrences; state detail once
cwickham Aug 26, 2026
c63c274
report.md: drop the redundant rule column
cwickham Aug 26, 2026
73847e1
Remove the systemic/localized label; drop the occurrence detail column
cwickham Aug 26, 2026
9edb955
findings.json: emit anchor-relative paths
cwickham Aug 26, 2026
20437ee
Self-ignore the _axe-checks artifact dir
cwickham Aug 26, 2026
8c04019
scan: collision-proof cell names, percent-encoded page URLs
cwickham Aug 26, 2026
d00b227
discover: two-mode marker matches the toggle definition, not its name
cwickham Aug 26, 2026
32ce785
axe action: exit through exitWithCleanup
cwickham Aug 26, 2026
01224ed
Add --fail-on: exit 1 on new findings by impact
cwickham Aug 27, 2026
e144a57
Await the site server's teardown before exiting
cwickham Aug 27, 2026
e1a2475
Close the v1 review's test gaps: flag parsing and exit codes
cwickham Aug 27, 2026
d7f36ec
Write llm-docs/axe-scan-architecture.md; repoint dead references
cwickham Aug 27, 2026
ef13b94
dev-docs/axe-scan.md: experimental how-to
cwickham Aug 27, 2026
86b9312
changelog: quarto dev-call axe (PR number pending)
cwickham Aug 27, 2026
f0717fd
report.md: backtick code spans instead of <code> raw HTML
cwickham Aug 27, 2026
191a63d
Move axe from dev-call to quarto call
cwickham Aug 27, 2026
f8e82fb
Test the browser-launch failure path
cwickham Aug 27, 2026
c1a2167
Don't dangle an empty colon when Chrome discovery finds nothing
cwickham Aug 27, 2026
f0261c7
Docs: cross-page grouping is a heuristic, not a law
cwickham Aug 27, 2026
257376c
Docs: post-render usage and a sticky PR comment
cwickham Aug 27, 2026
d07b536
Help text: say EXPERIMENTAL, not Prototype
cwickham Aug 27, 2026
c2b8cbf
changelog: fill in the axe PR number (#14815)
cwickham Aug 27, 2026
20e870c
Docs: how to read and render report.md
cwickham Aug 27, 2026
379c6ad
Default narrow viewport: 390x844 -> 320x568 (WCAG's reflow width)
cwickham Aug 27, 2026
d4a2de0
llm-doc: separate CDP transport from scan commands
cwickham Aug 31, 2026
5319ff6
Docs: say plainly that posting the report is workflow config
cwickham Sep 3, 2026
7d8768e
--max-pages counts scannable pages, not raw HTML files
cwickham Sep 3, 2026
ee7f633
Usage errors exit 3, not 1 or 2
cwickham Sep 3, 2026
06f0db9
Reject duplicate baseline signatures instead of merging them
cwickham Sep 3, 2026
5141d88
Skip the scan on an incremental project render
cwickham Sep 3, 2026
82da67d
Bound the timeout recovery, and stop the cell it abandoned
cwickham Sep 3, 2026
08742c1
Make the changelog's docs pointer a link; drop it from --help
cwickham Sep 3, 2026
b9c6852
axe: don't tell the reader an unseen baseline entry is resolved
cderv Sep 9, 2026
1741f3b
axe: reject a --report destination the scan owns
cderv Sep 9, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .claude/rules/dev-tools/axe-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
paths:
- "src/command/call/axe/**"
- "tests/unit/axe-*.test.ts"
- "tests/smoke/axe/**"
- "tests/docs/axe-scan/**"
---

# Axe Scan Command (`quarto call axe`)

For the scanner's architecture — pipeline, mode discovery, signatures,
baseline semantics, exit codes, and the rationale behind each — see
`llm-docs/axe-scan-architecture.md`.

For how to use the command (flags, baseline workflow, CI recipe), see
`dev-docs/axe-scan.md`.

The signature normalization scheme is pinned by
`tests/unit/axe-signature.test.ts`: changing `normalizeSelector` or
`signatureOf` in a way that re-keys signatures requires bumping
`kSignatureScheme` in `schemas.ts` on purpose.
22 changes: 22 additions & 0 deletions .claude/rules/formats/html-dark-mode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
paths:
- "src/format/html/format-html-info*"
- "src/format/html/format-html-bootstrap*"
- "src/command/render/pandoc-html*"
- "src/resources/formats/html/templates/quarto-html-*body.ejs"
- "src/core/brand/**"
- "src/command/dev-call/axe/**"
---

# HTML Dark Mode

`formatDarkMode(format)` (`src/format/html/format-html-info.ts`) is the single
predicate for "does this page have a dark mode" — `undefined` means no.
The `data-mode` link attribute measures compiled CSS darkness; it does not
declare the author's slot. The dark slot's stable identity is the
`quarto-color-alternate` class.

For the full picture — configuration surface (`theme`, `brand`/`_brand.yml`),
rendered DOM markers, programmatic mode switching, and the known traps
(light-only brand, light-colored dark slots, `theme: darkly`) — see
`llm-docs/html-dark-mode-architecture.md`.
201 changes: 201 additions & 0 deletions dev-docs/axe-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# Scanning a site with `quarto call axe`

`quarto call axe` scans a rendered Quarto site for accessibility
violations with axe-core. It drives headless Chrome over every page, at
desktop and mobile widths, in each colour mode the page ships. It groups
violations by root cause, compares them against a committed baseline, and
writes a report you can read, commit review comments from, or gate CI on.

**Status: experimental.** The command is hidden (it does not show in
`quarto call` help) and makes no stability promise — flags, artifact shapes,
and semantics can change between prereleases. This page is contributor-facing
documentation; it graduates to quarto.org when the command is unhidden (the
same path the `axe:` render option's docs took).

Prerequisite: a Chromium the scanner can find. `quarto install
chrome-headless-shell` is the reliable route; an installed system Chrome or
Edge also works.

## When to reach for it

The render-time `axe:` option checks the page you are looking at, in your
browser, while you author. This command checks the *site*: every page, both
viewports, light and dark, at audit time or in CI. Reach for it when you
want a site-wide inventory, a regression gate, or output an agent can work
through.

## First scan

```sh
quarto render
quarto call axe _site
```

The scan prints its matrix up front (`43 pages (40 light+dark, 3 default) ×
2 viewports — 166 cells`), then one line per cell, then a findings summary.
Artifacts land at the project root (the nearest `_quarto.yml` at or above
the site dir):

- `_axe-checks/findings.json` — every finding, machine-readable.
- `_axe-checks/report.md` — the human summary, GitHub-flavored markdown.
- `_axe-checks/README.md` — generated docs for the artifacts, including the
baseline how-to. Read this after a scan; it is written for whoever (or
whatever) has only the artifact directory in front of them.
- `_axe-checks/cells/` — raw axe output per page × viewport × mode cell.

`_axe-checks/` ignores itself (`.gitignore` with `*`): it is a disposable
snapshot, not a thing to commit.

### Reading report.md

The report is GitHub-flavored markdown, made to be read where markdown
already renders: your editor's preview, GitHub, or the sticky PR comment
from the CI recipe below. Rendering it is optional. For a standalone HTML
view:

```sh
quarto render _axe-checks/report.md
```
Comment thread
cderv marked this conversation as resolved.

then open `_axe-checks/report.html` in your browser. The output lands
beside the report, inside the self-ignoring artifact directory.

`quarto preview _axe-checks/report.md` does **not** work from inside a
project: `_`-prefixed directories are not project inputs, so preview stops
with `No output created by quarto render report.md`. Use render-then-open,
or pass `--report` a path inside your site source (see the flags table) to
render and preview the report as part of the site.
Comment thread
cderv marked this conversation as resolved.

A finding is one *root cause*, not one element: an alt-less image in a
shared include shows up as one finding with an instance count, not once per
page. Grouping keys on axe's element selector (normalized), so this holds
when the pages describe the element the same way — reliably true for
Quarto's own chrome and for repeated template output, and occasionally
wrong for an anonymous element whose surrounding DOM differs page to page
(axe then picks different selectors, and one cause splits into two
findings). Findings on many pages usually come from a shared source — a
template, the theme, Quarto's own chrome — and one fix clears them all.

## Flags

| flag | default | |
|---|---|---|
| `--pages <globs>` | all `*.html` | comma-separated site-relative globs |
| `--exclude <globs>` | — | skip globs, applied after `--pages` |
| `--max-pages <n>` | ∞ | deterministic cap (sorted, first n scannable pages — redirect stubs don't use up the cap) |
| `--viewports <WxH,...>` | `1440x900,320x568` | |
| `--themes <light,dark>` | `light,dark` | filters two-mode pages; one-mode pages always scan once |
| `--timeout <ms>` | `30000` | per-cell budget |
| `--settle <ms>` | `50` | extra delay after the page reports ready |
| `--fail-on <impact>` | off | exit 1 on new findings at/above `minor`/`moderate`/`serious`/`critical` |
| `--report <path>` | `_axe-checks/report.md` | put the report elsewhere, e.g. inside your site source — can't name the baseline, `findings.json`, `README.md`, `.gitignore`, or anything inside `cells/` |

The narrow default viewport is 320 CSS px — the width WCAG's reflow
criterion (SC 1.4.10) names, equivalent to 400% zoom on a 1280 px window —
so every rule runs against the reflowed mobile layout.

A subset scan (`--pages`, `--exclude`, `--max-pages`) says so loudly in
every artifact: its counts describe the subset, not the site.

## Running it as a post-render step

The command works as a project `post-render` script, so a full render scans
itself:

```yaml
project:
type: website
post-render:
- quarto call axe _site/
Comment thread
cderv marked this conversation as resolved.
```

The script runs from the project directory after the outputs are written,
so the relative site dir and the artifact anchor resolve exactly as they do
on the command line. Exit codes propagate: without `--fail-on`, findings
never fail the render; with `--fail-on`, a new finding at the threshold —
or an incomplete scan — fails `quarto render` itself, with the scan's error
line in the render output. This adds the scan's runtime to each full render.

Only *full* renders scan. Quarto runs post-render scripts on incremental
renders and on preview reloads too, and the scan skips both with a note and
exit 0 — a partly-rebuilt site would mix this render's output with the last
one's. Run `quarto render` with no file argument to scan.

## The baseline workflow

The first scan of a real site reports findings you will not fix today:
upstream defects, deferred best-practice items, known false positives. The
baseline is the committed ledger of those decisions — `_axe-baseline.json`
at the project root, beside (not inside) `_axe-checks/`.

1. Scan, then fix what you can from `report.md`.
2. For each finding you are accepting instead: copy it out of
`findings.json` into the baseline's `findings` array, trim it, and write
a `note` saying why. The generated `_axe-checks/README.md` documents the
entry shape and the scoping rules (`pages: []` accepts site-wide;
a listed `pages` re-alerts anywhere else).
3. Commit `_axe-baseline.json`.

From then on, reports separate **new** findings from **baselined** ones,
and only new findings can fail CI. A baselined finding that escalates in
impact, or shows up on a page outside its scope, re-alerts as new. Entries
a scan no longer sees are reported as stale — prune them by hand, but only
after a scan of the full matrix (every page, viewport and mode) in which
every cell completed. Narrowing any axis with `--pages`, `--viewports` or
`--themes` means the scan can't tell "fixed" from "not scanned", and neither
can a full scan that lost a cell; the report says which applies to it.

There is deliberately no `--update-baseline`: every entry exists because
someone wrote it and said why.

## Exit codes and the CI recipe

| exit | meaning |
|---|---|
| `0` | scan complete; no new findings at/above `--fail-on` (when given) |
| `1` | complete scan, new findings at/above the `--fail-on` threshold |
| `2` | scan incomplete — a cell timed out or errored, no browser, nothing to scan. Takes precedence over 1: an incomplete scan never reads as a pass |
| `3` | usage error — a flag value the command can't accept (`--fail-on serius`), or a filter that matches nothing (`--themes dark` on a site with no dark mode). A typo is not a result, so it gets its own code |

A minimal GitHub Actions gate:

```yaml
- uses: quarto-dev/quarto-actions/setup@v2
- run: quarto install chrome-headless-shell --no-prompt
- run: quarto render
- run: quarto call axe _site --fail-on serious
Comment thread
cderv marked this conversation as resolved.
```

`report.md` is GitHub-flavored markdown, so a workflow can post it straight
into the PR — no Quarto-side tooling needed. Use a sticky comment, which
updates in place on each push rather than adding one comment per run:

```yaml
- if: always()
uses: marocchino/sticky-pull-request-comment@v2
with:
header: axe
path: _axe-checks/report.md
```

(`if: always()` keeps the comment current when `--fail-on` fails the job;
`header` keys the comment so other sticky comments are untouched. GitHub
caps comment bodies at 65,536 characters; a whole-site report on a large
site can exceed it — trim or attach as an artifact instead.)

## Reading the output as an agent

Point the agent at `_axe-checks/` and let it read the generated `README.md`
first. Finding ids are stable across runs (`image-alt-6e3b76`), so "fix
`image-alt-6e3b76`" is a well-defined instruction, and each finding's
`occurrences[]` carries real selectors and HTML excerpts. Fixes belong in
Quarto *source* (`.qmd`, `_quarto.yml`, `_brand.yml`, theme `.scss`) — never
in the rendered site directory.
Comment thread
cderv marked this conversation as resolved.

## Where the pieces are documented

- How it works and why: `llm-docs/axe-scan-architecture.md`.
- What the artifacts mean, baseline entry shape: the generated
`_axe-checks/README.md` (regenerated every scan, always matches the build
that wrote it).
- Render-time single-page checking: the `axe:` HTML format option.
Loading
Loading