Skip to content

Spark, Databricks: parse the << and >> shift operators - #15

Merged
moshap-firebolt merged 1 commit into
firebolt/v0.63.0-patchesfrom
firebolt/spark-shift-operators
Oct 4, 2026
Merged

moshap-firebolt merged 1 commit into
firebolt/v0.63.0-patchesfrom
firebolt/spark-shift-operators

Conversation

@moshap-firebolt

@moshap-firebolt moshap-firebolt commented Oct 3, 2026 •

Copy link
Copy Markdown

Spark and Databricks both have the << and >> bitwise shift operators. The parser already supports them behind supports_bitwise_shift_operators; this turns that on for SparkSqlDialect and DatabricksDialect.

Spark's grammar binds a shift tighter than & and looser than +. The shared default puts shifts at the Caret level, which sits below &, so 1 & 3 << 1 would read as (1 & 3) << 1. Both dialects override get_next_precedence to place shifts just above Ampersand.

Tests in tests/sqlparser_spark.rs cover both operators, the operator tag, and the tree shape against & and +. The fork's full suite, fmt and clippy are clean.


Note

Low Risk
Dialect-only parser changes with targeted tests; no auth, data, or runtime execution paths affected.

Overview
Spark and Databricks now parse bitwise shift operators << and >> by enabling supports_bitwise_shift_operators on both dialects.

Both dialects override get_next_precedence so shifts bind tighter than & and looser than + (e.g. 1 & 3 << 1 → 1 & (3 << 1)), matching Spark’s grammar instead of the default precedence that would group shifts below &.

Spark tests in test_bitwise_shift_operators cover parsing, operator kinds, and precedence against & and +.

Reviewed by Cursor Bugbot for commit b4c500b. Bugbot is set up for automated code reviews on this repo. Configure here.

Both dialects have them. Spark's grammar binds a shift tighter than &
and looser than +, so the dialects place it just above Ampersand rather
than at the shared Caret level, which sits below &.
@moshap-firebolt
moshap-firebolt merged commit bd4770d into firebolt/v0.63.0-patches Oct 4, 2026
21 checks passed
@moshap-firebolt
moshap-firebolt deleted the firebolt/spark-shift-operators branch October 5, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant