Skip to content

end the zlib stream when readDataBlock and sync throw - #4035

Open
ArjunD3456 wants to merge 1 commit into
apache:mainfrom
ArjunD3456:zlib-stream-end-on-throw
Open

ArjunD3456 wants to merge 1 commit into
apache:mainfrom
ArjunD3456:zlib-stream-end-on-throw

Conversation

@ArjunD3456

Copy link
Copy Markdown

What is the purpose of the change

DataFileReaderBase::readDataBlock sets up a z_stream with inflateInit2 for every deflate-coded block it reads and only reaches the matching inflateEnd once the inflate loop has run to completion. When a block does not inflate, the throw inside that loop leaves the scope with the stream still live, so the ~7 KiB of state zlib allocated for it is lost. The payload comes straight from the file, so any corrupt or crafted container leaks it, and a process that keeps reading untrusted files and handles avro::Exception grows without bound. DataFileWriterBase::sync has the same shape around deflateInit2, and the resize call in either loop can leave the scope the same way. Giving the stream a small scope guard puts the inflateEnd/deflateEnd call on every exit path, which is how ZstdCompressWrapper and ZstdDecompressWrapper already release their contexts.

Verifying this change

This change added tests and can be verified as follows:

  • added testCorruptDeflateBlock to DataFileTests, which writes a deflate file, rewrites the first payload byte so the block header carries the reserved RFC 1951 BTYPE 3, and checks the read is rejected rather than decoded; ./build.sh test and ./build.sh lint both stay green
  • measured the leak: reading that file 2000 times and catching the exception each time grew the heap by 14336048 bytes (7168 per failed read) before this change and by 48 bytes after

Documentation

  • Does this pull request introduce a new feature? no

@github-actions github-actions Bot added the C++ Pull Requests for C++ binding label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C++ Pull Requests for C++ binding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant