Skip to content

ci: streamline workflow caching & manage ressources - #16287

Open
anastygnome wants to merge 1 commit into
uutils:mainfrom
anastygnome:ci/optimize-workflows
Open

anastygnome wants to merge 1 commit into
uutils:mainfrom
anastygnome:ci/optimize-workflows

Conversation

@anastygnome

Copy link
Copy Markdown
Contributor

Reduce CI cache churn and redundant workflow work.

Adds shared Rust caching with explicit namespaces and default-branch-only writes, limits matrix fan-out, removes redundant single-entry matrices, and filters localization runs.

Refs #10535
Refs #10347
Refs #9315

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from c500c24 to 593ca8e Compare November 8, 2026 17:25
@anastygnome
anastygnome marked this pull request as ready for review November 8, 2026 17:26
Copilot AI balanced review requested due to automatic review settings November 8, 2026 17:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome

anastygnome commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

cc: @oech3
cc: @cakebaker
cc: @xtqqczze

@anastygnome anastygnome changed the title ci: streamline workflow caching ci: streamline workflow caching & manage ressources Nov 8, 2026
@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 593ca8e to f3f7fd3 Compare November 8, 2026 20:28
Copilot AI balanced review requested due to automatic review settings November 8, 2026 20:28
@anastygnome
anastygnome marked this pull request as draft November 8, 2026 20:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 155.30 MB (+60 KB, +0.04%)

Significant per-binary changes:
  printenv     1.09 MB ->    1.15 MB  (+60 KB, +5.36%)

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from f3f7fd3 to 9df0d0c Compare November 8, 2026 21:37
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/env/printenv is no longer failing!

@oech3

oech3 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Can we share caches even if we don't have --target=host-tuple (target/(missing host tuple name at here)/debug/*)?

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch 6 times, most recently from 2d94cf7 to cc1b845 Compare November 9, 2026 09:52
@anastygnome

anastygnome commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor Author

Can we share caches even if we don't have --target=host-tuple (target/(missing host tuple name at here)/debug/*)?

Yep, done.

@anastygnome
anastygnome marked this pull request as ready for review November 9, 2026 09:57
Copilot AI balanced review requested due to automatic review settings November 9, 2026 09:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from cc1b845 to 59ad3a6 Compare November 9, 2026 10:32
Copilot AI balanced review requested due to automatic review settings November 9, 2026 10:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 59ad3a6 to ff29b8f Compare November 9, 2026 11:31
Copilot AI balanced review requested due to automatic review settings November 9, 2026 11:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

Copilot AI balanced review requested due to automatic review settings November 9, 2026 11:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 8075102 to 389ce4c Compare November 9, 2026 17:25
Copilot AI balanced review requested due to automatic review settings November 9, 2026 17:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 389ce4c to b1ee7bf Compare November 9, 2026 17:30
Copilot AI balanced review requested due to automatic review settings November 9, 2026 17:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

Comment thread .github/workflows/benchmarks.yml Outdated
- uses: ./.github/actions/rust-cache
with:
key: bench-${{ matrix.package }}-${{ matrix.type }}
save-if: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this key is per package with save-if false, even on main, so nothing ever writes it and the restore always misses, no?

with:
workspaces: "fuzz -> target"
key: fuzz-${{ matrix.target.name }}
save-if: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here: fuzz-<target> is never saved anywhere, so this cache is dead. Did you mean to reuse the fuzz-build key?

@anastygnome anastygnome Nov 9, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, fuzz-run now reuses fuzz-build (from the namespace) from the main branch

shell: bash
run: |
echo "RUSTC_WRAPPER=sccache" >> "$GITHUB_ENV"
echo "SCCACHE_GHA_ENABLED=true" >> "$GITHUB_ENV"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sccache still writes from every PR, and it creates far more cache entries than rust-cache. Should it be read-only outside main too?

@anastygnome anastygnome Nov 9, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SCCACHE_GHA_RW_MODE is now RW on the main branch & RO elsewhere

Comment thread .github/workflows/openbsd.yml Outdated
# Test building with make
if (test -z "\$FAULT"); then
echo "::group::cargo clean before make"
if ! cargo clean; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated to caching, please split. Also, why run cargo clean twice?

Comment thread .github/workflows/CICD.yml Outdated
DOCKER_OPTS: '--volume /etc/passwd:/etc/passwd --volume /etc/group:/etc/group'
strategy:
fail-fast: false
max-parallel: 6

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

21 jobs at 6 at a time will make every PR much slower. Do we really need this?

Comment thread .github/workflows/l10n.yml Outdated

on:
pull_request:
paths: &l10n_paths

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are some of these jobs required checks? With a paths filter they would stay pending on PRs that skip them.

Comment thread .github/workflows/fuzzing.yml Outdated
name: Fuzz (${{ matrix.target.name }})
needs: detect
if: needs.detect.outputs.targets != '[]'
needs: [detect, fuzz-build]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fuzz-build only saves on main, so on PRs this just makes fuzz-run wait for the build. Why the new dependency?

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from b1ee7bf to 4fc9f1c Compare November 9, 2026 21:28
Copilot AI balanced review requested due to automatic review settings November 9, 2026 21:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome

Copy link
Copy Markdown
Contributor Author

@sylvestre had forgotten to push one commit ... rebased

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 4fc9f1c to b638cba Compare November 9, 2026 21:31
Copilot AI balanced review requested due to automatic review settings November 9, 2026 21:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome
anastygnome requested a review from sylvestre November 9, 2026 21:36
@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from b638cba to 8b031f0 Compare November 9, 2026 22:00
Copilot AI balanced review requested due to automatic review settings November 9, 2026 22:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 8b031f0 to 8df0bd2 Compare November 10, 2026 09:21
Copilot AI balanced review requested due to automatic review settings November 10, 2026 09:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@anastygnome

Copy link
Copy Markdown
Contributor Author

@sylvestre lgtm

@anastygnome
anastygnome force-pushed the ci/optimize-workflows branch from 8df0bd2 to 66d577a Compare November 11, 2026 07:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants