You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An INNERPiecewiseMergeJoin reports a false output equivalence between the two sides of its range predicate, which can make the optimizer drop a required sort and return wrongly ordered results.
PiecewiseMergeJoinExec::compute_properties passes the join's on pair to join_equivalence_properties as if it were an equijoin key. For an INNER join, join_equivalence_properties registers left_on == right_on as an output equivalence. But PWMJ's on is a range predicate (l.v < r.v), not equality — so this equivalence is false. A downstream ORDER BY on the "other" column can then be optimized away because the planner believes it is already sorted.
To Reproduce
setdatafusion.optimizer.enable_piecewise_merge_join = true;
createtablel(v int) asvalues (1),(2),(3),(5),(8);
createtabler(v int) asvalues (4),(6),(9),(2);
selectl.vas lv, r.vas rv
from l join r onl.v<r.vwherel.v=2order byr.v;
Describe the bug
An
INNERPiecewiseMergeJoinreports a false output equivalence between the two sides of its range predicate, which can make the optimizer drop a required sort and return wrongly ordered results.PiecewiseMergeJoinExec::compute_propertiespasses the join'sonpair tojoin_equivalence_propertiesas if it were an equijoin key. For anINNERjoin,join_equivalence_propertiesregistersleft_on == right_onas an output equivalence. But PWMJ'sonis a range predicate (l.v < r.v), not equality — so this equivalence is false. A downstreamORDER BYon the "other" column can then be optimized away because the planner believes it is already sorted.To Reproduce
Result with PiecewiseMergeJoin:
The correct result (what
NestedLoopJoinreturns with the flag off) is ordered byrv:The plans show why — PWMJ sorts only on
l.v, dropping ther.vsort:Expected behavior
A range join adds no column equivalences.
PiecewiseMergeJoinshould matchNestedLoopJoin; theORDER BY r.vsort must be preserved.Additional context
INNERis affected —join_equivalence_properties/EquivalenceGroup::joinonly registerson-pair equalities forInner.PiecewiseMergeJoinis experimental (enable_piecewise_merge_joindefaults tofalse), but this is worth fixing before default-enablement.PiecewiseMergeJoinwork in Datafusion #17427. Fix + regression test coming.