Repository navigation
fix: harden release reruns and pin checks - #507
Conversation
Port elixir-cloud-aai/trs-filer#311: - publish: new step "Check release state" right after "Check chart", before Docker login, attestation, signing and tagging. It runs `release_assets.py precheck` (a published release must already have this run's chart and its Sigstore bundle) and, for a published release, requires the `X.Y.Z` image tag to point to the run's digest, so a re-run of all jobs fails before it moves any tag. - check_release_pins.py: also reject `[tool.uv.sources]` overrides, package indexes other than https://pypi.org/simple, `uv.lock` packages from other sources (the app itself excepted) and a missing `uv.lock`. - check_release_pins.py: treat empty chart `dependencies: []` or `{}` as no dependencies (as in trs-filer), so the script is identical in all four repos again. - Tests and docs (release.yml header, docs/development.md). Closes #506 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The `gh` stand-in of `test_release_state_check_runs` returned the releases for any arguments, so a "Check release state" step that queried the wrong repository or endpoint still passed. It now answers only the step's request (`repos/$GH_REPO/releases?per_page=100`), and `test_release_state_checked_first` checks that the step takes `GH_REPO` from the run. Found by Sourcery on elixir-cloud-aai/drs-filer#135. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideImplementationExtend release pin validation to reject non-PyPI uv sources, indexes and lockfile packages, and treat empty chart dependency lists or maps as having no dependencies.
Add a precheck mode that verifies published release assets before signing, while retaining the existing post-signing release-state check.
ConfigurationAdd an early release-state and image-digest check so published-release reruns fail before Docker login, attestation, signing or tagging.
DocumentationDocument the early release checks and rerun behavior, and explain the expanded release pin requirements.
TestsAdd coverage for empty chart dependencies, uv source and lockfile validation, and lockfile error cases.
Test precheck behavior for published assets, missing bundles, unexpected assets and unpublished releases.
Test that the workflow runs release checks before publishing actions and rejects unsafe reruns or digest mismatches.
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #507 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 37 37
Lines 3475 3544 +69
=========================================
+ Hits 3475 3544 +69 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Linked issuesChecked at commit f7909c5 #506 fix: harden release reruns and pin checks · 4 of 5 met · 1 to check by hand
|
There was a problem hiding this comment.
Hey - I've found 5 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path=".github/workflows/release.yml" line_range="405" />
<code_context>
+ # on a re-run of failed jobs, which reuses both. Drafts and new
+ # releases pass; "Create release" checks the release again (keep the
+ # fetch of the releases the same in both steps)
+ - name: Check release state
+ env:
+ GH_TOKEN: ${{ github.token }}
</code_context>
<issue_to_address>
**Published release points to wrong image**
When a release is published after the precheck snapshot but before `Tag image` runs, the workflow uses the stale precheck result, so `Tag image` moves `X.Y.Z` (and possibly `latest`) to this run’s digest; the later `Create release` check happens too late to prevent the published release from pointing at a different image.
Re-fetch and validate the release state and image digest before moving tags, and abort tagging if the release has been published or its digest differs.
Also at `.github/workflows/release.yml:414-415`, `.github/workflows/release.yml:418`.
</issue_to_address>
### Comment 2
<location path="scripts/check_release_pins.py" line_range="110" />
<code_context>
+UV_SOURCE_KINDS = ("git", "url", "path", "workspace")
+# Settings of `[tool.uv]` that name package indexes (or, `find-links`,
+# other places to find packages), each a URL or a list of URLs
+UV_INDEX_SETTINGS = ("index-url", "extra-index-url", "find-links")
</code_context>
<issue_to_address>
**Non-PyPI default index passes the release check**
When `[tool.uv].default-index` is configured with a non-PyPI URL while locked packages remain sourced from PyPI, `UV_INDEX_SETTINGS` omits `default-index`, so `_uv_pins` reports no finding and `_lock_pins` passes; the release check accepts the non-PyPI configuration.
Include `default-index` in `UV_INDEX_SETTINGS` so `_uv_pins` checks it.
Also at `scripts/check_release_pins.py:111`.
</issue_to_address>
### Comment 3
<location path="scripts/check_release_pins.py" line_range="139" />
<code_context>
item_indent = None
- if in_dependencies and value.strip():
+ # An empty list (or map) has no dependencies to check
+ if in_dependencies and value.strip() not in ("", "[]", "{}"):
dependencies.append(None)
continue
</code_context>
<issue_to_address>
**Valid empty dependencies block releases**
When `Chart.yaml` uses an empty flow-style dependency collection with internal whitespace, such as `[ ]` or `{ }`, `_chart_dependencies` recognizes only the exact strings `[]` and `{}`, so it appends `None`; `find_dev_pins` reports that dependencies cannot be checked and blocks release readiness.
Normalize whitespace inside the flow-style value before checking whether the collection is empty.
</issue_to_address>
### Comment 4
<location path="tests/test_workflows.py" line_range="695-697" />
<code_context>
+ ),
+ "docker": (
+ '[[ "$*" == "buildx imagetools inspect $IMAGE_NAME:1.2.3 '
+ '--format {{ json .Manifest }}" ]] && '
+ + (f'cat "{tmp_path}/manifest.json"' if tagged else "exit 1")
+ ),
+ }
+ for name, body in stubs.items():
</code_context>
<issue_to_address>
**Wrong inspection command passes test**
When the `new-digest` case runs and the workflow invokes `docker` with unexpected arguments, `test_release_state_check_runs` treats this as the expected inspection failure; its nonzero-status assertion and generic “release a new patch version” check pass even when the command arguments are wrong, so the test does not verify the digest-mismatch branch.
Make the test distinguish an unexpected invocation from an unavailable tag, and assert the mismatch-specific output for the `new-digest` case.
</issue_to_address>
### Comment 5
<location path="scripts/check_release_pins.py" line_range="139" />
<code_context>
item_indent = None
- if in_dependencies and value.strip():
+ # An empty list (or map) has no dependencies to check
+ if in_dependencies and value.strip() not in ("", "[]", "{}"):
dependencies.append(None)
continue
</code_context>
<issue_to_address>
**Empty chart dependencies misdescribed**
The module docstring still says flow-style `[...]` and `{...}` dependencies cannot be read, but `_chart_dependencies` now accepts empty `[]` and `{}`. Users relying on that description get an incorrect account of which charts the release check rejects.
Clarify that only non-empty flow-style dependencies are reported as unreadable.
</issue_to_address>Sourcery assessment
Approval pending. 4 findings to address first.
Blocking findings: .github/workflows/release.yml:405, scripts/check_release_pins.py:110, scripts/check_release_pins.py:139, tests/test_workflows.py:697
In `test_release_state_check_runs`, the `new-digest` case passed for any failure that says "release a new patch version", which the error of a failed image inspection says too. A wrong `docker` invocation in "Check release state" therefore went unnoticed. The case now requires the digest-mismatch error (`points to <digest>`). Found by Sourcery on #507. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The docstrings of `check_release_pins.py` still said that chart
dependencies in flow style can't be read, but an empty list or map
(`[]`, `{}`) now counts as no dependencies. Found by Sourcery on
#507.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In `test_release_state_check_runs`, the `new-digest` case passed for any failure that says "release a new patch version", which the error of a failed image inspection says too. A wrong `docker` invocation in "Check release state" therefore went unnoticed. The case now requires the digest-mismatch error (`points to <digest>`). Found by Sourcery on elixir-cloud-aai/protes#507. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The docstrings of `check_release_pins.py` still said that chart
dependencies in flow style can't be read, but an empty list or map
(`[]`, `{}`) now counts as no dependencies. Found by Sourcery on
elixir-cloud-aai/protes#507.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix: harden release reruns and pin checks Port elixir-cloud-aai/trs-filer#311: - publish: new step "Check release state" right after "Check chart", before Docker login, attestation, signing and tagging. It runs `release_assets.py precheck` (a published release must already have this run's chart and its Sigstore bundle) and, for a published release, requires the `X.Y.Z` image tag to point to the run's digest, so a re-run of all jobs fails before it moves any tag. - check_release_pins.py: also reject `[tool.uv.sources]` overrides, package indexes other than https://pypi.org/simple, `uv.lock` packages from other sources (the app itself excepted) and a missing `uv.lock`. - Tests and docs (release.yml header, docs/development.md). Closes #131 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: check the release lookup of the gh stub The `gh` stand-in of `test_release_state_check_runs` returned the releases for any arguments, so a "Check release state" step that queried the wrong repository or endpoint still passed. It now answers only the step's request (`repos/$GH_REPO/releases?per_page=100`), and `test_release_state_checked_first` checks that the step takes `GH_REPO` from the run. Found by Sourcery on #135. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: check the digest error of the release step In `test_release_state_check_runs`, the `new-digest` case passed for any failure that says "release a new patch version", which the error of a failed image inspection says too. A wrong `docker` invocation in "Check release state" therefore went unnoticed. The case now requires the digest-mismatch error (`points to <digest>`). Found by Sourcery on elixir-cloud-aai/protes#507. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: describe empty flow-style chart deps The docstrings of `check_release_pins.py` still said that chart dependencies in flow style can't be read, but an empty list or map (`[]`, `{}`) now counts as no dependencies. Found by Sourcery on elixir-cloud-aai/protes#507. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Closes #506 (part of elixir-cloud-aai/.github#92). This ports elixir-cloud-aai/trs-filer#311 (fix for trs-filer#310) as is, plus trs-filer's earlier handling of empty chart
dependencies:publishhas a new step, "Check release state", right after "Check chart" and "Set up Docker Buildx" and before "Log in to Docker Hub", "Attest build provenance", "Sign image" and "Tag image". It runsscripts/release_assets.py precheck <tag>, which works likestatebut also requires a published release to have the Sigstore bundle of each asset already. For a published release, it also requiresdocker buildx imagetools inspect "${IMAGE_NAME}:${VERSION}"to point to the run'sDIGEST. Otherwise the step fails and says to release a new patch version. As a result, a re-run of all jobs of a published release no longer movesX.Y.Z/latestto a new digest. Re-running only the failed jobs still passes, and drafts and new releases are unchanged.scripts/check_release_pins.pynow also rejects the following.https://pypi.org/simpleis the only allowed index.[tool.uv.sources]entries of these kinds:git,url,path/editable,workspace, a non-PyPI or undefinedindex, and unknown forms[[tool.uv.index]],index-url,extra-index-url,find-linksuv.lockpackages whose source is not PyPI (the app itself is exempt)uv.lockdependencies: []or{}inChart.yamlnow counts as no dependencies, as in trs-filer. Before, it was reported as an unreadable flow-style dependency. New test:test_chart_dependencies_empty.With this,
scripts/release_assets.pyandscripts/check_release_pins.pyare identical to trs-filer's again, apart from one trs-filer-specific comment.The workflow test is also slightly stricter than trs-filer's. Its
ghstand-in answers only the step's own releases request (repos/$GH_REPO/releases?per_page=100), and the test checks the step'sGH_REPO. Sourcery found this gap on elixir-cloud-aai/drs-filer#135, which ports the same change.Docs: the header comment of
release.yml, the step's own comment, and "Versioning and releases" indocs/development.md(the two release-state checks, re-runs, and the new pin-check findings).Effect on
dev: none.devpins released FOCA and foca-plugins, everyuv.lockpackage exceptprotesitself comes from PyPI, andpython scripts/check_release_pins.pyexits 0.tests/test_check_release_pins.py::test_release_checkoutstill skips on non-release checkouts.Verification
coverage run -m pytest -W ignore::DeprecationWarning, thencoverage report: total coverage 100%, and all tests outside the Helm tests passed (1969 passed, 11 skipped). The 211 failures are all intests/test_chart.py(209) andtests/test_config_examples.py::test_chart_examples_valid(2). Both need the library chart built withhelm dependency build, which isn't available on this machine. CI runs them.pytest tests/test_check_release_pins.py tests/test_release_assets.py tests/test_workflows.py: 228 passed, 1 skipped (test_release_checkout, not a release checkout).test_chart_dependencies_emptyfailed before the fix (3 cases) and passes after it.pre-commit run --all-filespassed: ruff check/format, semgrep and its rule tests, gitleaks, actionlint with shellcheck, ast-grep and its rule tests, mypy, and pre-commit-hooks.tests/test_workflows.py::test_release_state_check_runsruns the step's script against stand-ins forghanddocker. Cases: no release, draft, published with the run's digest (passes), published with another digest or without an image tag (fails), published with another chart (fails), no bundle, and mutable release.DOCKERHUB_ORGset on the fork. No release is planned now.f7909c5(run): all checks are green, and Integration (TESK), Push image and Publish image were skipped, as intended on PRs. On6cc3703, before the Sourcery fixes, the same set passed: Lint, Docs, both Test jobs, Chart, Smoke test, Integration (label, service info), Scan image, Release readiness, CI passed, CodeQL, Codecov and Read the Docs.Review: one round of adversarial and quality review (full lane, since this changes the release workflow). Neither found anything high or medium.
Sourcery opened five threads, all now answered and resolved:
bee9c40: thenew-digestcase of the workflow test now requires the digest-mismatch error. Before, any failure passed it, including a failed inspection caused by wrongdockerarguments. When I changed the step's--formatargument as a test, the case failed.f7909c5: the docstrings ofcheck_release_pins.pynow say that an empty[]or{}counts as no dependencies.[tool.uv] default-index. uv 0.12.13 rejects that key inpyproject.tomlas an unknown field. The default index there is an[[tool.uv.index]]entry, which is already checked, and an index set on the command line shows up inuv.lock.[ ]and{ }with inner spaces. The check fails closed with a readable finding, this chart uses block style, and the script's parsing should change in all four repos at once.Both fixes are also in drs-filer#135 and cloud-registry#419.
Declined review findings
precheckfails first and gives the remedy that matches theimmutableflag. In practice it can't be reached.docs/development.mdexplains theprecheckasset rule only in the script and the step comment. Declined: same as trs-filer, and the user-facing section already describes the effect.Open questions
None.
🤖 Generated with Claude Code