Skip to content

fix: handle non-UTF-8 packed refs - #2216

Closed
Sohel2309 wants to merge 1 commit into
gitpython-developers:mainfrom
Sohel2309:fix/2064-packed-refs-encoding
Closed

fix: handle non-UTF-8 packed refs#2216
Sohel2309 wants to merge 1 commit into
gitpython-developers:mainfrom
Sohel2309:fix/2064-packed-refs-encoding

Conversation

@Sohel2309

@Sohel2309 Sohel2309 commented Aug 14, 2026

Copy link
Copy Markdown

Fixes #2064.

Bug

GitPython raises UnicodeDecodeError when iterating .git/packed-refs if a packed ref name contains bytes that are not valid UTF-8.

Git ref names are byte strings and are not guaranteed to contain valid UTF-8. A single non-UTF-8 ref should not make the entire packed-refs file unreadable.

Root Cause

SymbolicReference._iter_packed_refs() opened packed-refs in text mode with strict UTF-8 decoding:

open(..., "rt", encoding="UTF-8")

This caused UnicodeDecodeError when a packed ref contained non-UTF-8 bytes.

Fix

Read packed-refs in binary mode and decode each line using surrogateescape.

This follows the existing lenient byte-to-string handling used elsewhere in GitPython and preserves the original bytes during round-tripping instead of raising an exception.

Regression Test

Added a regression test covering a packed ref containing a non-UTF-8 byte and verifying that iterating packed refs does not raise UnicodeDecodeError.

Validation

  • Targeted regression test: passed
  • test/test_refs.py: regression test passed
  • ruff check: passed
  • ruff format --check: passed
  • git diff --check: passed

@Byron

Byron commented Aug 30, 2026

Copy link
Copy Markdown
Member

I don't think that making this pass is the right choice here.
Instead, GitPython shouldn't assume encodings at all, or at most ascii, and get its typing straight.
A lot of work will be needed to fix this, and I plan to remove all the pure python implementations in favor of using the Git binary instead.
Thanks for your understanding.

@Byron Byron closed this Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Encoding issue with tags in packed-refs file

2 participants