[tree] Fix bulk read inflating the reported branch size - #22949
Open
guitargeek wants to merge 1 commit into
Open
[tree] Fix bulk read inflating the reported branch size#22949guitargeek wants to merge 1 commit into
guitargeek wants to merge 1 commit into
Conversation
Test Results 23 files 23 suites 3d 15h 40m 51s ⏱️ For more details on these failures, see this check. Results for commit bf8a530. ♻️ This comment has been updated with latest results. |
vepadulano
reviewed
Jul 29, 2026
Calling `GetBulkRead().GetBulkEntries()/GetEntriesSerialized()` on a branch increased its reported Total Size by 4 bytes on every call, so the size was not invariant with respect to the API used to read the branch. Root cause: a bulk read loads a basket into fBaskets via `TObjArray::AddAt` (`TBranch::GetBasketImpl`), which raises the array's fLast. The bulk-IO path then disassociates that basket from the array with a plain slot assignment, `fBaskets[fReadBasket] = nullptr`. That is not the inverse of `AddAt:` the non-const `TObjArray::operator[]` does `fLast = std::max(j, GetAbsLast()`), so writing the null actually pins `fLast` at `fReadBasket` rather than lowering it. The array is then streamed (by `GetTotalSize()`/`Print()`, and on a subsequent `TTree::Write()`) with one extra trailing slot, and `TObjArray::Streamer` emits an additional 4-byte null-pointer marker for it. Fix: drop the basket with `fBaskets.RemoveAt(fReadBasket)` at the three bulk-IO disassociation sites (GetBasketAndFirst, GetBulkEntries, GetEntriesSerialized). RemoveAt is the proper inverse of the AddAt done when the basket was loaded: it clears the slot and walks fLast back down past trailing nulls, matching what DropBaskets already does. Notes for review: - Fix location. The alternative is to make TObjArray::Streamer / GetTotalSize tolerant of trailing null slots. Fixing the read path was chosen instead so that a bulk read leaves no observable state change at all, which is what the issue asks for, and it is narrower than teaching the streamer to compact. - Performance. Bulk IO is a hot path, but this adds no meaningful cost: the previous operator[] assignment already acquired gCoreMutex (a conditional read guard, a no-op unless thread-safety is enabled) and already called Changed(). The only delta is read-guard -> write-guard. - Scope. Other sites null a slot the same inflating way and share the latent bug (the fBaskets[i] = nullptr in the flush path, and the fBaskets.AddAt(nullptr, idx) calls in the basket-unload paths). They are intentionally left untouched here: they are unrelated code paths not exercised by this issue, and are better addressed separately. - RemoveAt is a no-op on an already-null slot, so there is no double-remove hazard when GetBulkEntries runs after GetBasketAndFirst cleared the slot. The disassociated basket is not leaked; it is still parked in fExtraBasket. Adds a regression test (`BulkApiTest.SizeInvariantAfterBulkRead`) asserting `GetTotalSize()` is invariant across a bulk read; it fails without the fix. Closes root-project#8961 🤖 Done with the help of AI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Calling
GetBulkRead().GetBulkEntries()/GetEntriesSerialized()on a branch increased its reported Total Size by 4 bytes on every call, so the size was not invariant with respect to the API used to read the branch.Root cause: a bulk read loads a basket into fBaskets via
TObjArray::AddAt(TBranch::GetBasketImpl), which raises the array's fLast. The bulk-IO path then disassociates that basket from the array with a plain slot assignment,fBaskets[fReadBasket] = nullptr. That is not the inverse ofAddAt:the non-constTObjArray::operator[]doesfLast = std::max(j, GetAbsLast()), so writing the null actually pinsfLastatfReadBasketrather than lowering it. The array is then streamed (byGetTotalSize()/Print(), and on a subsequentTTree::Write()) with one extra trailing slot, andTObjArray::Streameremits an additional 4-byte null-pointer marker for it.Fix: drop the basket with
fBaskets.RemoveAt(fReadBasket)at the three bulk-IO disassociation sites (GetBasketAndFirst, GetBulkEntries, GetEntriesSerialized). RemoveAt is the proper inverse of the AddAt done when the basket was loaded: it clears the slot and walks fLast back down past trailing nulls, matching what DropBaskets already does.Notes for review:
Fix location. The alternative is to make TObjArray::Streamer / GetTotalSize tolerant of trailing null slots. Fixing the read path was chosen instead so that a bulk read leaves no observable state change at all, which is what the issue asks for, and it is narrower than teaching the streamer to compact.
Performance. Bulk IO is a hot path, but this adds no meaningful cost: the previous operator[] assignment already acquired gCoreMutex (a conditional read guard, a no-op unless thread-safety is enabled) and already called Changed(). The only delta is read-guard -> write-guard.
Scope. Other sites null a slot the same inflating way and share the latent bug (the fBaskets[i] = nullptr in the flush path, and the fBaskets.AddAt(nullptr, idx) calls in the basket-unload paths). They are intentionally left untouched here: they are unrelated code paths not exercised by this issue, and are better addressed separately.
RemoveAt is a no-op on an already-null slot, so there is no double-remove hazard when GetBulkEntries runs after GetBasketAndFirst cleared the slot. The disassociated basket is not leaked; it is still parked in fExtraBasket.
Adds a regression test (
BulkApiTest.SizeInvariantAfterBulkRead) assertingGetTotalSize()is invariant across a bulk read; it fails without the fix.Closes #8961
🤖 Done with the help of AI.