fix(bq_driver): fix Unsupported arrow data type (0) crash for INTERVAL types in HTAPI - #1636
Open
Anshu6250 wants to merge 1 commit into
Open
fix(bq_driver): fix Unsupported arrow data type (0) crash for INTERVAL types in HTAPI#1636Anshu6250 wants to merge 1 commit into
Anshu6250 wants to merge 1 commit into
Conversation
| std::make_tuple("RangeIntervalTestTable", | ||
| "SELECT * FROM " | ||
| "`bigquery-devtools-drivers.DATATYPERANGETEST." | ||
| "RangeIntervalTestTable` LIMIT 10000") |
Collaborator
There was a problem hiding this comment.
Have the limit as 100000
Collaborator
Author
There was a problem hiding this comment.
changed to 1 million
Anshu6250
force-pushed
the
arrow_fix_2
branch
2 times, most recently
from
August 18, 2026 05:10
6ebb8af to
f5381d1
Compare
Anshu6250
changed the title
shivamd-gpartner
approved these changes
Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes an
Unsupported arrow data type (0)crash that occurs when querying BigQueryINTERVALdata types with HTAPI enabled. Previously, the driver lacked the necessary schema mappings and string formatting for Arrow's specific interval memory layouts. This issue was consistently causing query failures in third-party applications like Excel, PowerBI, and Tableau, specifically when loading theAllDataTypes_2andRangeIntervalTestTabletables.Changes Included
GetArrowSchemato explicitly recognize Arrow's interval memory layouts (INTERVAL_MONTHS,INTERVAL_DAY_TIME, andINTERVAL_MONTH_DAY_NANO) and maps them to the driver's internalBQDataType::kInterval. Furthermore, it adds dedicated handling for each of these layouts withinProcessRecordBatch. This ensures the underlying integer values—such as years, months, days, hours, minutes, seconds, and nanoseconds—are accurately extracted and formatted into the exact Y-M D H:M:S[.fraction] string structure expected by our downstream data translation layers.perf pipeline : link