Skip to content

feat(devpkg): configurable Nix binary cache via DEVBOX_NIX_BINARY_CACHE#2891

Open
abueide wants to merge 2 commits into
jetify-com:mainfrom
abueide:configurable-nix-binary-cache
Open

feat(devpkg): configurable Nix binary cache via DEVBOX_NIX_BINARY_CACHE#2891
abueide wants to merge 2 commits into
jetify-com:mainfrom
abueide:configurable-nix-binary-cache

Conversation

@abueide

@abueide abueide commented Jun 23, 2026

Copy link
Copy Markdown

Summary

Fixes #2599.

Devbox hardcodes https://cache.nixos.org as the Nix binary cache it queries for prebuilt package outputs. This makes devbox run fail in network-restricted environments (such as enterprise CI runners with no public internet egress), even when an internal mirror that serves the identical store paths is configured as a Nix substituter.

The hardcoded URL (internal/devpkg/narinfo_cache.go) is used in two places:

  1. The narinfo precheckreadCaches seeds its cache list with this URL, and FillNarInfoCache runs unconditionally at the start of every devbox run (via flakePlan). fetchNarInfoStatusFromHTTP does a 5s HEAD to <cache>/<hash>.narinfo; on a network error it returns the error, and fetchNarInfoStatusOnce propagates it.
  2. The fromStore of the builtins.fetchClosure expressions in the generated flake.nix.

So on a locked-down runner, devbox run fails hard with:

Error: error running script "..." in Devbox: Head "https://cache.nixos.org/<hash>.narinfo": context deadline exceeded

…even when the toolchain is fully available from an internal substituter. Setting substituters in nix.conf does not help: builtins.fetchClosure contacts its fromStore directly and ignores substituters, and the Go-level precheck has its own hardcoded URL.

This PR adds a DEVBOX_NIX_BINARY_CACHE environment variable that overrides the default cache, following the same pattern as the existing DEVBOX_SEARCH_HOST override. When unset, behavior is unchanged (https://cache.nixos.org), so this is fully backwards compatible. Pointed at an internal mirror that proxies cache.nixos.org, devbox run then works with no public egress:

export DEVBOX_NIX_BINARY_CACHE="https://nix-cache.internal.example.com/mirror"
devbox run ...

Changes

  • envir: add a DevboxNixBinaryCache (DEVBOX_NIX_BINARY_CACHE) constant to the central env-var registry.
  • devpkg: replace the binaryCache constant with a BinaryCache() accessor that resolves the env var via envir.GetValueOrDefault, defaulting to https://cache.nixos.org; readCaches uses it.
  • shellgen: the generated flake's fromStore now uses the http(s) cache the output was actually found in, via a new fetchClosureStore template helper that falls back to the configured binary cache for s3:// caches fetchClosure cannot use directly.

How was it tested?

go test ./internal/devpkg/... ./internal/shellgen/...
go tool golangci-lint run --timeout 5m
  • New unit tests: TestBinaryCache (internal/devpkg) covers the default and the override; TestFetchClosureStore (internal/shellgen) covers http(s) pass-through, the s3 fallback, and the configured fallback.
  • Existing golden-file flake tests are unchanged when the variable is unset.
  • Verified end-to-end: with DEVBOX_NIX_BINARY_CACHE set, the generated flake.nix emits fromStore = "<mirror>" and the narinfo precheck targets the mirror; unset, it remains https://cache.nixos.org.

Community Contribution License

All community contributions in this pull request are licensed to the project
maintainers under the terms of the
Apache 2 License.

By creating this pull request, I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 License as stated in
the
Community Contribution License.

Devbox hardcodes https://cache.nixos.org as the binary cache it queries for
prebuilt package outputs. This URL is used in two places:

  1. The narinfo HEAD precheck in devpkg.readCaches / fetchNarInfoStatusFromHTTP,
     which runs unconditionally on every `devbox run` (via FillNarInfoCache).
  2. The fromStore of the builtins.fetchClosure expressions in the generated
     flake.nix.

In a network-restricted environment (e.g. an enterprise CI runner with no
public egress) cache.nixos.org is unreachable, so the precheck HEAD request
times out. Because fetchNarInfoStatusFromHTTP returns the error and
fetchNarInfoStatusOnce propagates it, `devbox run` fails hard with
`Head "https://cache.nixos.org/<hash>.narinfo": context deadline exceeded`
even when an internal mirror that serves the identical store paths is
available and configured as a Nix substituter. (Setting `substituters` in
nix.conf does not help: builtins.fetchClosure contacts fromStore directly and
ignores substituters.)

Add a DEVBOX_NIX_BINARY_CACHE environment variable that overrides the default
cache, following the same pattern as the existing DEVBOX_SEARCH_HOST override
(envir.GetValueOrDefault). When unset, behavior is unchanged
(https://cache.nixos.org), so this is fully backwards compatible.

The override flows to both consumers: readCaches seeds its list with the
resolved cache, and the flake template now sets fromStore to the http(s) cache
the output was actually found in (via a new fetchClosureStore template helper),
falling back to the configured binary cache for s3 caches that fetchClosure
cannot use directly.

Pointed at an internal mirror that proxies cache.nixos.org, `devbox run` then
works with no public egress.
Copilot AI review requested due to automatic review settings June 23, 2026 23:32
@abueide abueide changed the title devpkg, shellgen: make the Nix binary cache configurable (DEVBOX_NIX_BINARY_CACHE) Make the Nix binary cache configurable via the DEVBOX_NIX_BINARY_CACHE env var Jun 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes Devbox’s Nix binary cache configurable via DEVBOX_NIX_BINARY_CACHE, replacing the previously hardcoded https://cache.nixos.org in both the Go-level narinfo precheck and the generated flake.nix builtins.fetchClosure fromStore value. This enables devbox run to work in network-restricted environments when an internal mirror/proxy is available.

Changes:

  • Introduces envir.DevboxNixBinaryCache and devpkg.BinaryCache() (defaulting to https://cache.nixos.org, overridable via env var).
  • Updates narinfo cache discovery to seed read caches from devpkg.BinaryCache() instead of a constant.
  • Updates flake generation to set fromStore based on the cache URI actually used, with tests added for both BinaryCache() and fetchClosureStore() behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
internal/shellgen/tmpl/flake.nix.tmpl Uses fetchClosureStore to dynamically set builtins.fetchClosure.fromStore.
internal/shellgen/generate.go Adds fetchClosureStore template helper and wires it into template functions.
internal/shellgen/generate_internal_test.go Adds unit tests for fetchClosureStore.
internal/envir/env.go Registers the new DEVBOX_NIX_BINARY_CACHE env var constant with documentation.
internal/devpkg/narinfo_cache.go Adds BinaryCache() and uses it to seed cache URIs instead of a hardcoded constant.
internal/devpkg/narinfo_cache_test.go Adds unit tests for BinaryCache() default/override behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/shellgen/generate_internal_test.go
Comment thread internal/shellgen/tmpl/flake.nix.tmpl Outdated
@abueide abueide changed the title Make the Nix binary cache configurable via the DEVBOX_NIX_BINARY_CACHE env var feat(devpkg): configurable Nix binary cache via DEVBOX_NIX_BINARY_CACHE Jun 23, 2026
Address review feedback:
- Render the fetchClosure fromStore through a nixString helper that escapes
  backslash, double quote, and the ${ antiquotation, so a configured cache URL
  (DEVBOX_NIX_BINARY_CACHE) with special characters can't break flake
  evaluation or be interpreted as Nix interpolation. Output is unchanged for
  ordinary URLs.
- TestFetchClosureStore now always sets DEVBOX_NIX_BINARY_CACHE (empty when
  unset) so it can't be affected by the caller's environment.
- Add TestNixString.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Airgap setup with nix substituters not working

2 participants