chore: implement phase 10 by Tofel · Pull Request #2788 · smartcontractkit/chainlink-testing-framework

Tofel · 2026-09-01T11:20:09Z

Wires the watch/check subcommands to the gate library, with a table + JSON renderer and H6/H7 exit-code mapping. Extends Result with per-rule/global thresholds and a real skew bound, exports SkewHardLimit, and rejects --states normal.

Review focus: table.go rendering, main.go subcommand wiring, and the exit-code mapping.

github-actions · 2026-09-01T11:21:21Z

⚠️ API Diff Results - `github.com/smartcontractkit/chainlink-testing-framework/grafana-alertcheck`

⚠️ Breaking Changes (1)

`package github` (1)

com/smartcontractkit/chainlink-testing-framework/grafana-alertcheck/cmd/grafana-alertcheck — 🗑️ Removed

✅ Compatible Changes (1)

`package github` (1)

com/smartcontractkit/chainlink-testing-framework/grafana-alertcheck/cmd — ➕ Added

📄 View full apidiff report

Copilot

🟡 Changes recommended

The check command can still exit 0/1 even when table rendering fails, and there is a misleading comment in skew/bound selection logic that should be corrected.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds the phase-10 CLI surfaces for the Grafana alert gate by wiring the new watch (record) and check (classify) subcommands to the internal gate library, including a human-readable table renderer + optional JSON output, updated result payload fields (thresholds + skew bound), and the documented H6/H7 exit-code mapping.

Changes:

Export SkewHardLimit and extend gate.Result with per-rule/global thresholds and a skew bound to support richer CLI output.
Implement grafana-alertcheck watch and grafana-alertcheck check subcommands, plus shared flag parsing helpers and validation (including rejecting --states normal).
Add §20.2 table rendering + golden tests, and add tests for CLI flag validation and exit-code mapping.

File summaries

File	Description
grafana-alertcheck/internal/gate/source.go	Switches skew hard-limit usage to exported `SkewHardLimit`.
grafana-alertcheck/internal/gate/schedule.go	Exports `SkewHardLimit` constant for CLI reporting.
grafana-alertcheck/internal/gate/classify.go	Extends `Result` with thresholds + skew bound and populates them during decide/classify.
grafana-alertcheck/internal/gate/check.go	Updates library comments to reflect renderer behavior for Violation.Note.
grafana-alertcheck/cmd/grafana-alertcheck/main.go	Wires `watch`/`check` into subcommand dispatch and updates usage text.
grafana-alertcheck/cmd/grafana-alertcheck/common.go	Adds shared flag registration and parsing helpers (alerts, states, preexisting).
grafana-alertcheck/cmd/grafana-alertcheck/watch.go	Implements `watch` CLI including daemon-child dispatch.
grafana-alertcheck/cmd/grafana-alertcheck/watch_test.go	Adds watch CLI flag-validation + daemon-child dispatch tests.
grafana-alertcheck/cmd/grafana-alertcheck/check.go	Implements `check` CLI including table + optional JSON output and exit-code mapping.
grafana-alertcheck/cmd/grafana-alertcheck/check_test.go	Adds exit-code mapping and check CLI validation tests.
grafana-alertcheck/cmd/grafana-alertcheck/table.go	Implements §20.2 human table renderer (rules, violations, footer).
grafana-alertcheck/cmd/grafana-alertcheck/table_test.go	Golden-style deterministic rendering tests for the table output.

Review details

Files reviewed: 12/12 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal.

* chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 6 Invariant defended: H2. The one question: can watch return success over a window that nothing is recording? Watch() records the first observation of each non-skipped rule, then detaches a child that polls at the cadence in the header. The parent returns only after the child reports ready on an inherited pipe, and writes the pidfile after that. A clean stop writes the sentinel; a hard error does not. * chore: add a unit test, remove build tags * chore: address code review comments * chore: implement phase 7 (#2785) * chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 5 Add the JSONL evidence log (P5). - log.go: Header/Poll records, reduction, H2 transition markers, §3.2 verification, append-only Writer with flock, ReadLog - flock_unix.go: non-blocking exclusive lock, unix only - schedule.go: DeriveTimingsFromLog — log-mode cadence comes from the header, never from the definitions * chore: remove unix build tags * chore: address code review comments * chore: implement phase 6 (#2784) * chore: implement phase 6 Invariant defended: H2. The one question: can watch return success over a window that nothing is recording? Watch() records the first observation of each non-skipped rule, then detaches a child that polls at the cadence in the header. The parent returns only after the child reports ready on an inherited pipe, and writes the pidfile after that. A clean stop writes the sentinel; a hard error does not. * chore: add a unit test, remove build tags * chore: address code review comments * chore: implement phase 7 (#2785) * chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 4 Add per-rule poll timings, scheduler, and budget check (P4). - schedule.go: DeriveTimings, Scheduler, CheckBudget (§5) - Address review: add Folder/Title resolve test, rename CheckBudget's minPollEvery to tightestUID * chore: enhance unit tests * chore: address code review comments * chore: implement phase 5 (#2783) * chore: implement phase 5 Add the JSONL evidence log (P5). - log.go: Header/Poll records, reduction, H2 transition markers, §3.2 verification, append-only Writer with flock, ReadLog - flock_unix.go: non-blocking exclusive lock, unix only - schedule.go: DeriveTimingsFromLog — log-mode cadence comes from the header, never from the definitions * chore: remove unix build tags * chore: address code review comments * chore: implement phase 6 (#2784) * chore: implement phase 6 Invariant defended: H2. The one question: can watch return success over a window that nothing is recording? Watch() records the first observation of each non-skipped rule, then detaches a child that polls at the cadence in the header. The parent returns only after the child reports ready on an inherited pipe, and writes the pidfile after that. A clean stop writes the sentinel; a hard error does not. * chore: add a unit test, remove build tags * chore: address code review comments * chore: implement phase 7 (#2785) * chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 3 Add Resolve() for alert name resolution (uid:/Title/Folder/Title/ Folder/Group/Title forms, UID collapse, no-match suggestions) and the grafana-alertcheck CLI's list subcommand, the first runnable piece of the gate. Incorporates review fixes: reject empty path segments in classifyForm, guard uid: against an empty suffix, scope the no-match rule count and suggestions to supported rule kinds only, and exit 0 on -h/--help. * chore: enhance unit tests * chore: implement phase 4 (#2782) * chore: implement phase 4 Add per-rule poll timings, scheduler, and budget check (P4). - schedule.go: DeriveTimings, Scheduler, CheckBudget (§5) - Address review: add Folder/Title resolve test, rename CheckBudget's minPollEvery to tightestUID * chore: enhance unit tests * chore: address code review comments * chore: implement phase 5 (#2783) * chore: implement phase 5 Add the JSONL evidence log (P5). - log.go: Header/Poll records, reduction, H2 transition markers, §3.2 verification, append-only Writer with flock, ReadLog - flock_unix.go: non-blocking exclusive lock, unix only - schedule.go: DeriveTimingsFromLog — log-mode cadence comes from the header, never from the definitions * chore: remove unix build tags * chore: address code review comments * chore: implement phase 6 (#2784) * chore: implement phase 6 Invariant defended: H2. The one question: can watch return success over a window that nothing is recording? Watch() records the first observation of each non-skipped rule, then detaches a child that polls at the cadence in the header. The parent returns only after the child reports ready on an inherited pipe, and writes the pidfile after that. A clean stop writes the sentinel; a hard error does not. * chore: add a unit test, remove build tags * chore: address code review comments * chore: implement phase 7 (#2785) * chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 2 Fix retry-error conflation, measure full poll latency, and harden Source test doubles for concurrency. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 3 (#2781) * chore: implement phase 3 Add Resolve() for alert name resolution (uid:/Title/Folder/Title/ Folder/Group/Title forms, UID collapse, no-match suggestions) and the grafana-alertcheck CLI's list subcommand, the first runnable piece of the gate. Incorporates review fixes: reject empty path segments in classifyForm, guard uid: against an empty suffix, scope the no-match rule count and suggestions to supported rule kinds only, and exit 0 on -h/--help. * chore: enhance unit tests * chore: implement phase 4 (#2782) * chore: implement phase 4 Add per-rule poll timings, scheduler, and budget check (P4). - schedule.go: DeriveTimings, Scheduler, CheckBudget (§5) - Address review: add Folder/Title resolve test, rename CheckBudget's minPollEvery to tightestUID * chore: enhance unit tests * chore: address code review comments * chore: implement phase 5 (#2783) * chore: implement phase 5 Add the JSONL evidence log (P5). - log.go: Header/Poll records, reduction, H2 transition markers, §3.2 verification, append-only Writer with flock, ReadLog - flock_unix.go: non-blocking exclusive lock, unix only - schedule.go: DeriveTimingsFromLog — log-mode cadence comes from the header, never from the definitions * chore: remove unix build tags * chore: address code review comments * chore: implement phase 6 (#2784) * chore: implement phase 6 Invariant defended: H2. The one question: can watch return success over a window that nothing is recording? Watch() records the first observation of each non-skipped rule, then detaches a child that polls at the cadence in the header. The parent returns only after the child reports ready on an inherited pipe, and writes the pidfile after that. A clean stop writes the sentinel; a hard error does not. * chore: add a unit test, remove build tags * chore: address code review comments * chore: implement phase 7 (#2785) * chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

* chore: implement phase 1 Strict parsers for the state and ruler endpoints (H1), a Prometheus-style duration parser, and fixtures sliced from real Grafana 13.1.0 payloads covering every required/optional-field and must-error case, including the "Normal (NoData)"/"Normal (Error)" composite reason states found live in the current fleet capture (not in the original plan's vocabulary). * chore: apply code review comments * chore: implement phase 2 (#2780) * chore: implement phase 2 Fix retry-error conflation, measure full poll latency, and harden Source test doubles for concurrency. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 3 (#2781) * chore: implement phase 3 Add Resolve() for alert name resolution (uid:/Title/Folder/Title/ Folder/Group/Title forms, UID collapse, no-match suggestions) and the grafana-alertcheck CLI's list subcommand, the first runnable piece of the gate. Incorporates review fixes: reject empty path segments in classifyForm, guard uid: against an empty suffix, scope the no-match rule count and suggestions to supported rule kinds only, and exit 0 on -h/--help. * chore: enhance unit tests * chore: implement phase 4 (#2782) * chore: implement phase 4 Add per-rule poll timings, scheduler, and budget check (P4). - schedule.go: DeriveTimings, Scheduler, CheckBudget (§5) - Address review: add Folder/Title resolve test, rename CheckBudget's minPollEvery to tightestUID * chore: enhance unit tests * chore: address code review comments * chore: implement phase 5 (#2783) * chore: implement phase 5 Add the JSONL evidence log (P5). - log.go: Header/Poll records, reduction, H2 transition markers, §3.2 verification, append-only Writer with flock, ReadLog - flock_unix.go: non-blocking exclusive lock, unix only - schedule.go: DeriveTimingsFromLog — log-mode cadence comes from the header, never from the definitions * chore: remove unix build tags * chore: address code review comments * chore: implement phase 6 (#2784) * chore: implement phase 6 Invariant defended: H2. The one question: can watch return success over a window that nothing is recording? Watch() records the first observation of each non-skipped rule, then detaches a child that polls at the cadence in the header. The parent returns only after the child reports ready on an inherited pipe, and writes the pidfile after that. A clean stop writes the sentinel; a hard error does not. * chore: add a unit test, remove build tags * chore: address code review comments * chore: implement phase 7 (#2785) * chore: implement phase 7 Invariant defended: H3. The one question: can a rule be called alive because it looked alive one poll ago? proveCoverage (grafana-alertcheck/internal/gate/coverage.go) is the pure coverage function: nine checks over one rule's polls — sentinel, from-bounds, heartbeat continuity, health error/nodata, liveness, in-window pause, rule absence, KeepLast. Liveness is absolute, never a delta. Cross-domain comparisons translate by each poll's own skew and widen boundary segments by its skew bound, fail-closed. * chore: enhance unit tests * chore: address code review comments * chore: implement phase 8 (#2786) * chore: implement phase 8 Invariant defended: H6/H7. The one question: can a violation ever outrank an unobservable rule, or can a pass happen without Violations empty and err nil? Adds classify.go: the pure per-instance classifier (outcome table, preexisting policy, BadFor) and decide(), the seam combining proveCoverage with those timelines under one Policy. Consolidates rule-poll filtering and skew translation onto pollsForRule/runnerTime, shared with coverage.go. * chore: rename some vars + add unit tests * chore: address code review comments * chore: implement phase 9 (#2787) * chore: implement phase 9 Invariant defended: H5/H7. The one question: can check report a pass over a window it did not prove? Check() is the I/O shell around the pure decide(). Single-step synthesizes the header and its own sentinel, so no mode flag reaches the pure layer. Log mode stops the recorder before the one full read. The header, not a definition re-resolved after the window closed, is the authority for what was paused when the window opened — it decides `skipped`, the drain set, and the transitionGrace max. The flock, not the pidfile, is the authority for whether a writer still exists. * chore: remove unix build tag * chore: implement phase 10 (#2788) * Wire watch/check subcommands to the gate library, with a table+JSON renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal. * chore: fix goreleaser.yaml and add version command * chore: implement phase 11 (#2789) * chore: implement phase 11 Add coverage.go's declared-KeepLast check (no_data_state/exec_err_state, not just an observed reason) and close the remaining §22 gaps: newly_bad's no-early-exit clock assertion, a recorder-mode gap right after the deploy, a rule's own coverage gap overriding its own recovery, a genuinely skew-discriminating staleness test, exit-2 consequences on two Reason-only coverage tests, and end-to-end checks for log-name collapse, a truncated log, and the real watch-written state histogram. * chore: address code review comments * chore: more concise comments (#2790) * chore: more concise comments * fix: merge conflict * chore: shorten comments * fix: resolve conflict * chore: use testify's require in tests (#2792) * chore: use testify's require in tests * chore: move remaining assumptions to testify * chore: address code review comments * chore: fix logging and std out printing (#2798) * chore: fix logging and std out printing * chore: truncate to seconds when comparing from time * chore: add centralized docs (#2802) * chore: add centralized docs * chore: further update docs * chore: address code review comments (#2807) * chore: address code review comments * chore: get rid of goreleaser

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch from da537a7 to 99e5355 Compare September 1, 2026 11:25

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch 2 times, most recently from 6da4834 to ebd9dc5 Compare September 1, 2026 14:18

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch from 99e5355 to 95381e3 Compare September 1, 2026 14:18

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch from ebd9dc5 to 73c5106 Compare September 1, 2026 14:46

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch from 95381e3 to 04ba08e Compare September 1, 2026 14:46

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch 2 times, most recently from 5e23350 to ae817a4 Compare September 2, 2026 09:45

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch 2 times, most recently from 7d9cc1e to effe464 Compare September 2, 2026 09:57

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch 4 times, most recently from dd089cd to 7019986 Compare September 4, 2026 15:03

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch 2 times, most recently from 1377467 to f659144 Compare September 4, 2026 15:15

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch from 7019986 to 66c67db Compare September 4, 2026 15:15

Tofel changed the title ~~Wire watch/check subcommands to the gate library, with a table+JSON~~ chore: implement phase 10 Sep 4, 2026

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch from f659144 to c61028a Compare September 7, 2026 09:35

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch from 66c67db to 621a0fa Compare September 7, 2026 09:35

Tofel marked this pull request as ready for review September 7, 2026 09:37

Tofel requested a review from a team as a code owner September 7, 2026 09:37

Copilot AI lite review requested due to automatic review settings September 7, 2026 09:37

product-security-plaid-production Bot requested a review from skudasov September 7, 2026 09:37

Copilot started reviewing on behalf of Tofel September 7, 2026 09:37 View session

Copilot AI reviewed Sep 7, 2026

View reviewed changes

Comment thread grafana-alertcheck/cmd/grafana-alertcheck/check.go Outdated

Comment thread grafana-alertcheck/internal/gate/classify.go Outdated

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch from c61028a to ab74eba Compare September 7, 2026 09:46

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch from 621a0fa to a7612c5 Compare September 7, 2026 09:46

Wire watch/check subcommands to the gate library, with a table+JSON

df1d58a

renderer and H6/H7 exit-code mapping. Extend Result with per-rule/global thresholds and a real skew bound; export SkewHardLimit; reject --states normal.

chore: fix goreleaser.yaml and add version command

a6afb46

Tofel force-pushed the dx-5122-alerts-assertion-p10 branch from a7612c5 to a6afb46 Compare September 7, 2026 15:02

Tofel force-pushed the dx-5122-alerts-assertion-p9 branch from ab74eba to 7c3f206 Compare September 7, 2026 15:02

Tofel removed this pull request from stack #2791 September 9, 2026 09:52

Tofel closed this Sep 9, 2026

Tofel reopened this Sep 9, 2026

Tofel merged commit f2264e1 into dx-5122-alerts-assertion-p9 Sep 9, 2026
50 of 57 checks passed

Tofel deleted the dx-5122-alerts-assertion-p10 branch September 9, 2026 09:58

Navigation Menu

Uh oh!

chore: implement phase 10 - #2788

chore: implement phase 10#2788
Tofel merged 3 commits into
dx-5122-alerts-assertion-p9from
dx-5122-alerts-assertion-p10

Tofel commented Sep 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Tofel commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ API Diff Results - github.com/smartcontractkit/chainlink-testing-framework/grafana-alertcheck

⚠️ Breaking Changes (1)

package github (1)

✅ Compatible Changes (1)

package github (1)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Tofel commented Sep 1, 2026 •

edited

Loading

github-actions Bot commented Sep 1, 2026 •

edited

Loading

⚠️ API Diff Results - `github.com/smartcontractkit/chainlink-testing-framework/grafana-alertcheck`

`package github` (1)

`package github` (1)