Skip to content

fix(serve): reset HF_HUB_OFFLINE, not TRANSFORMERS_OFFLINE, after a DJL build - #6434

Open
AkshayShah03 wants to merge 1 commit into
aws:masterfrom
AkshayShah03:fix/djl-hf-hub-offline-reset
Open

AkshayShah03 wants to merge 1 commit into
aws:masterfrom
AkshayShah03:fix/djl-hf-hub-offline-reset

Conversation

@AkshayShah03

Copy link
Copy Markdown

Issue #, if available: N/A

Description of changes:

_build_for_djl sets HF_HUB_OFFLINE=1 for local modes. After creating the model, it resets the flag, but on the wrong key:

if "HF_HUB_OFFLINE" in self.env_vars:
    self.env_vars.update({"TRANSFORMERS_OFFLINE": "0"})

This looks like a leftover from the flag rename; the TGI, HF and TEI builders all reset HF_HUB_OFFLINE itself. Since env_vars persists on the ModelBuilder, a build() for LOCAL_CONTAINER followed by a build() for SAGEMAKER_ENDPOINT deployed the endpoint with HF_HUB_OFFLINE=1, plus a stray TRANSFORMERS_OFFLINE=0. The DJL container then can't download HF_MODEL_ID from the Hub.

This change resets HF_HUB_OFFLINE, as the other builders do. Local mode is unaffected: the model, and therefore the local container, is created before the reset, and the container definition copies env_vars.

Testing:

  • New test_local_build_does_not_leave_hf_hub_offline_for_endpoint_build builds for local and then for the endpoint, capturing the env at each _create_model call. It fails before this change.
  • test_local_mode_sets_hf_hub_offline asserted the leftover in-memory state. It now checks the env the model is created with, which is still HF_HUB_OFFLINE=1.
  • Server builder tests: no new failures. The same 15 tests fail on master.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…JL build

_build_for_djl sets HF_HUB_OFFLINE=1 for local modes and then, after
creating the model, reset TRANSFORMERS_OFFLINE instead of HF_HUB_OFFLINE
(a leftover from the flag rename). The TGI, HF and TEI builders all reset
HF_HUB_OFFLINE. Because env_vars persists on the ModelBuilder, building
for LOCAL_CONTAINER and then for SAGEMAKER_ENDPOINT deployed the endpoint
with HF_HUB_OFFLINE=1 (plus a stray TRANSFORMERS_OFFLINE=0), so the DJL
container could not download HF_MODEL_ID from the Hub.

Reset HF_HUB_OFFLINE like the other builders. The model is created before
the reset (the container definition copies env_vars), so local mode still
runs offline; the existing test now checks the env at model creation.

This branch is waiting to be deployed

1 waiting deployment
manual-approval — f4c4d04b Waiting Oct 12, 2026 by AkshayShah03 via wait-for-approval #1988
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant