Skip to content

chore(storage): Error handling for open and unlinked files - #35031

Open
shubhangi-google wants to merge 3 commits into
googleapis:mainfrom
shubhangi-google:fix_file_upload_unliked_files
Open

chore(storage): Error handling for open and unlinked files#35031
shubhangi-google wants to merge 3 commits into
googleapis:mainfrom
shubhangi-google:fix_file_upload_unliked_files

Conversation

@shubhangi-google

@shubhangi-google shubhangi-google commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Fixes issue where uploading unlinked temporary files (such as large payloads handled by the Puma web server) fails during the CRC32c digest computation.
closes #34987

@shubhangi-google
shubhangi-google force-pushed the fix_file_upload_unliked_files branch from bc63dcf to 92e3408 Compare July 28, 2026 06:47
@shubhangi-google
shubhangi-google marked this pull request as ready for review July 28, 2026 07:06
@shubhangi-google
shubhangi-google requested review from a team and yoshi-approver as code owners July 28, 2026 07:06
@@ -63,33 +63,35 @@ def self.crc32c_for local_file
# Computes a base64-encoded digest for a local file or IO stream.
#
# This method handles two types of inputs for `local_file`:
# 1. A file path (String or Pathname): It efficiently streams the file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we removing this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are swapping the order of Case 1 and Case 2 to match the updated logic below. We need to check if the object is an IO-like stream (i.e., responds to :read) before checking if it's a file path. If we check for the file path first, an unlinked Tempfile will still return true for :to_path, but attempting to open that path will raise an Errno::ENOENT error. Checking if we can read it directly (:read) first fixes this.

# reading to ensure its position is not permanently changed.
# 2. A file path (String or Pathname): It efficiently streams the file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is true

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted to previous version

# Case 1: Input is a file path (String, Pathname, or object that responds to :to_path).
::File.open Pathname(local_file).to_path, "rb" do |f|
digest_class.file(f).base64digest
if local_file.respond_to?(:read)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we removing existing code?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are swapping the existing code to check for the :read capability (the IO-stream case) before falling back to the :to_path capability (the file path case).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error during CRC check when uploading an open + unlinked file

2 participants