Skip to content

fix: Windows large file support and binary-safe downloads - #248

Open
jiuker wants to merge 4 commits into
minio:mainfrom
jiuker:fix-windows-large-file-compat
Open

fix: Windows large file support and binary-safe downloads#248
jiuker wants to merge 4 commits into
minio:mainfrom
jiuker:fix-windows-large-file-compat

Conversation

@jiuker

@jiuker jiuker commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

fix #208

Fixes two Windows (LLP64) issues reported via the VCPKG port: binary file corruption on download and 32-bit 'long' overflow for files larger than 2 GiB.

Binary-safe downloads:

  • DownloadObject: open the temp output stream with std::ios::binary. On Windows the default text mode translates newlines and treats 0x1A as EOF, corrupting downloaded binary content (e.g. zip archives).

Large file support (LLP64: 'long' is strictly 32-bit on Windows):

  • SelectResult (types.h) and the parser in select.cc: bytes_scanned / bytes_processed / bytes_returned switch from 'long int' to 'long long', and std::stol to std::stoll
  • StatObject (baseclient.cc): parse the content-length header with std::stoll so resp.size (size_t) is not truncated above 2 GiB
  • ComposeSource (args.h/args.cc): object_size_ member and its assignment switch from 'long' to 'long long' so composing sources larger than 2 GiB does not overflow

No behavioral change on LP64 platforms (Linux/macOS), where 'long' is already 64-bit.

Files changed: include/miniocpp/args.h, include/miniocpp/types.h, src/args.cc, src/baseclient.cc, src/client.cc, src/select.cc

Summary by CodeRabbit

  • Bug Fixes
    • Improved support for large object sizes and Select-operation metrics with expanded numeric range handling.
    • Enhanced parsing of large content-length and event metric values.
    • Empty metric values are now represented as unavailable instead of misleading placeholder values.
    • Fixed object downloads to reliably write data in binary format across platforms.

Fixes two Windows (LLP64) issues reported via the VCPKG port:
binary file corruption on download and 32-bit 'long' overflow for files
larger than 2 GiB.

Binary-safe downloads:
- DownloadObject: open the temp output stream with std::ios::binary. On
  Windows the default text mode translates newlines and treats 0x1A as EOF,
  corrupting downloaded binary content (e.g. zip archives).

Large file support (LLP64: 'long' is strictly 32-bit on Windows):
- SelectResult (types.h) and the parser in select.cc: bytes_scanned /
  bytes_processed / bytes_returned switch from 'long int' to 'long long',
  and std::stol to std::stoll
- StatObject (baseclient.cc): parse the content-length header with
  std::stoll so resp.size (size_t) is not truncated above 2 GiB
- ComposeSource (args.h/args.cc): object_size_ member and its assignment
  switch from 'long' to 'long long' so composing sources larger than 2 GiB
  does not overflow

No behavioral change on LP64 platforms (Linux/macOS), where 'long' is
already 64-bit.

Files changed: include/miniocpp/args.h, include/miniocpp/types.h,
src/args.cc, src/baseclient.cc, src/client.cc, src/select.cc
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@jiuker, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 42 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c7e07a33-6fab-46e1-88d1-820a704cd4af

📥 Commits

Reviewing files that changed from the base of the PR and between a788fe3 and 1d9fe92.

📒 Files selected for processing (1)
  • tests/tests.cc

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f141675d-d0b1-4411-a608-9b78a6740b81

📥 Commits

Reviewing files that changed from the base of the PR and between 9e2ce43 and a788fe3.

📒 Files selected for processing (4)
  • include/miniocpp/args.h
  • include/miniocpp/types.h
  • src/args.cc
  • src/select.cc

📝 Walkthrough

Walkthrough

The change replaces sentinel-based size values with optional types, uses 64-bit parsing for object and Select metrics, and opens temporary download files in binary mode.

Changes

Large-file and binary compatibility

Layer / File(s) Summary
Optional object-size handling
include/miniocpp/args.h, src/args.cc, src/baseclient.cc
ComposeSource::object_size_ uses std::optional<size_t>. Header construction and initialization checks use optional-value presence. StatObject parses content-length with std::stoll.
Optional Select metrics
include/miniocpp/types.h, src/select.cc
SelectResult metrics and Select event counters use std::optional<long long>. Empty XML values remain unset, and present values use std::stoll.
Binary temporary downloads
src/client.cc
Temporary download files open with std::ios::binary.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: ⚪ Minimal · up to a788f

The PR fixes Windows binary downloads and large-file handling; the SelectResult API/ABI representation change should be documented for consumers, but no actionable merge-blocking risk remains.

Suggested reviewers: harshavardhana

Poem

A rabbit counts each byte with care,
Wide values travel everywhere.
Empty fields stay clearly bare,
Binary files keep their form,
Hop safely through the Windows storm.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the Windows large-file and binary-download fixes in the changeset.
Linked Issues check ✅ Passed The changes implement the linked issue requirements for binary-safe downloads and 64-bit file-size parsing and storage [#208].
Out of Scope Changes check ✅ Passed All changes directly support the linked issue objectives and no unrelated code changes are evident.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jiuker

jiuker commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@include/miniocpp/args.h`:
- Line 385: Replace ComposeSource::object_size_’s -1 sentinel with
std::optional<size_t>, initialize it empty, and set it when BuildHeaders
succeeds. Update ComposeSource::ObjectSize() to check has_value() and return the
stored size through the existing size_t API without signed-to-unsigned
conversion.

In `@include/miniocpp/types.h`:
- Around line 296-298: Update include/miniocpp/types.h at lines 296-298 and
305-309 to include <optional> and change the three public Select metric fields
and corresponding constructor parameters to std::optional<long long>. Update
src/select.cc lines 198-212 to leave each metric unset when its XML text is
empty and assign the parsed long long value otherwise.

In `@src/select.cc`:
- Around line 198-212: Add focused test coverage for SelectHandler::DataFunction
by supplying a synthetic Stats event whose BytesScanned, BytesProcessed, and
BytesReturned values all exceed INT32_MAX, then assert the callback receives the
exact long long values. Keep the test independent of large objects and ensure it
would detect any regression from std::stoll to std::stol, particularly on
Windows.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ced86126-1a6c-4cd9-9d7c-2a8a831768b9

📥 Commits

Reviewing files that changed from the base of the PR and between e764b82 and 9e2ce43.

📒 Files selected for processing (6)
  • include/miniocpp/args.h
  • include/miniocpp/types.h
  • src/args.cc
  • src/baseclient.cc
  • src/client.cc
  • src/select.cc

Comment thread include/miniocpp/args.h Outdated
Comment thread include/miniocpp/types.h Outdated
Comment thread src/select.cc Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Windows LLP64 compatibility: 2GB file limit (long vs int64_t) and missing std::ios::binary

1 participant