Skip to content

fix(server): close tokenizer config before building chat handler - #2422

Open
cloudgpuapp wants to merge 2 commits into
abetlen:mainfrom
cloudgpuapp:fix/tokenizer-config-close-r23
Open

cloudgpuapp wants to merge 2 commits into
abetlen:mainfrom
cloudgpuapp:fix/tokenizer-config-close-r23

Conversation

@cloudgpuapp

Copy link
Copy Markdown

hf-tokenizer-config passes json.load(open(path)) directly to the chat-handler factory. On invalid JSON, retaining the parsing exception also retains the json.load frame and its open file. Successful parsing relies on implicit finalization and emits an unclosed-file ResourceWarning on CPython.

Read the config in a with block, close it before building the handler, and pass the same parsed dictionary onward. File-open, JSON and handler errors still propagate; the default text encoding is unchanged. This is separate from model-proxy free() and metadata-override parsing.

Validation on macOS arm64:

  • Same four regression cases on original source: 3 failed / 1 passed; patched: 4 passed on Python 3.12 and Python 3.10.
  • Uses actual temporary files and json.load, including a retained JSONDecodeError traceback. Tests check warnings on success and handler failure, and preserve the missing-file error.
  • Real server/model.py and server/settings.py are imported. The local runner replaces the native package entry and auxiliary native modules at import; model construction and the handler factory are mocked. It does not load a compiled backend or model. The committed test file uses normal package imports.
  • Full make lint and git diff --check pass; patch applicability checked against current main.
  • Normal-import native tests, full model-dependent suite, HTTP serving and GPU inference were not run. In a built checkout with server/test extras, run python -m pytest tests/test_server_tokenizer_config_file.py -q.

Prepared with AI assistance. No provider integration or promotional link is added.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant