-
Notifications
You must be signed in to change notification settings - Fork 573
chore(storage): Error handling for open and unlinked files #35031
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -63,33 +63,35 @@ def self.crc32c_for local_file | |
| # Computes a base64-encoded digest for a local file or IO stream. | ||
| # | ||
| # This method handles two types of inputs for `local_file`: | ||
| # 1. A file path (String or Pathname): It efficiently streams the file | ||
| # to compute the digest without loading the entire file into memory. | ||
| # 2. An IO-like stream (e.g., File, StringIO): It reads the stream's | ||
| # content to compute the digest. The stream is rewound before and after | ||
| # 1. An IO-like stream (e.g., File, Tempfile, StringIO): It reads the | ||
| # stream's content in chunks to compute the digest without loading | ||
| # the entire file into memory. The stream is rewound before and after | ||
| # reading to ensure its position is not permanently changed. | ||
| # 2. A file path (String or Pathname): It efficiently streams the file | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure if this is true
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. reverted to previous version |
||
| # to compute the digest without loading the entire file into memory. | ||
| # | ||
| # @param local_file [String, Pathname, IO] The local file path or IO | ||
| # stream for which to compute the digest. | ||
| # @param digest_class [Class] The digest class to use for the | ||
| # calculation (e.g., `Digest::MD5`). It must respond to `.file` and | ||
| # `.base64digest`. | ||
| # calculation (e.g., `Digest::MD5`). It must respond to `.new`, | ||
| # `.file` and `.base64digest`. | ||
| # | ||
| # @return [String] The base64-encoded digest of the file's content. | ||
| # | ||
| def self._digest_for local_file, digest_class | ||
|
|
||
| if local_file.respond_to?(:to_path) || local_file.is_a?(String) | ||
| # Case 1: Input is a file path (String, Pathname, or object that responds to :to_path). | ||
| ::File.open Pathname(local_file).to_path, "rb" do |f| | ||
| digest_class.file(f).base64digest | ||
| if local_file.respond_to?(:read) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why are we removing existing code?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we are swapping the existing code to check for the :read capability (the IO-stream case) before falling back to the :to_path capability (the file path case). |
||
| # Case 1: Input is an open stream (File, StringIO, ActionDispatch::Http::UploadedFile, etc.). | ||
| local_file.rewind if local_file.respond_to?(:rewind) | ||
| digest = digest_class.new | ||
| while (chunk = local_file.read(16 * 1024)) | ||
| digest.update(chunk) | ||
| end | ||
| else | ||
| # Case 2: Input is an open stream (File or StringIO). | ||
| local_file.rewind | ||
| digest = digest_class.base64digest local_file.read | ||
| local_file.rewind | ||
| digest | ||
| local_file.rewind if local_file.respond_to?(:rewind) | ||
| digest.base64digest | ||
| elsif local_file.respond_to?(:to_path) || local_file.is_a?(String) | ||
| # Case 2: Input is a file path (String, Pathname). | ||
| digest_class.file(Pathname(local_file).to_path).base64digest | ||
| end | ||
| end | ||
| end | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we removing this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we are swapping the order of Case 1 and Case 2 to match the updated logic below. We need to check if the object is an IO-like stream (i.e., responds to :read) before checking if it's a file path. If we check for the file path first, an unlinked Tempfile will still return true for :to_path, but attempting to open that path will raise an Errno::ENOENT error. Checking if we can read it directly (:read) first fixes this.