Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
492b603
feat(physical-plan): adaptive conjunct reordering in FilterExec (comp…
adriangb Jun 28, 2026
500e154
feat(physical-plan): pool adaptive measurements across partition streams
adriangb Jun 28, 2026
f0d12f7
feat(physical-plan): use compact-once only in service of a reorder
adriangb Jun 28, 2026
b3cb96d
fix(physical-plan): address review feedback on adaptive filter
adriangb Jul 2, 2026
28a29d5
fix(physical-plan): reset adaptive filter state on re-execution
adriangb Jul 3, 2026
9ae8660
fix(physical-plan): harden adaptive filter edge cases
adriangb Jul 10, 2026
940a229
test(sqllogictest): drive adaptive filter across the settle boundary
adriangb Jul 10, 2026
8c53cdc
docs: tighten adaptive filter behavior claims
adriangb Jul 10, 2026
69564c8
refactor(physical-plan): FilterExec owns the enable gate; trace per-b…
adriangb Jul 10, 2026
76f5053
refactor(physical-plan): drop redundant adaptive filter epoch
adriangb Sep 7, 2026
f7b31d9
feat(physical-plan): report adaptive filter reordering in FilterExec …
adriangb Sep 7, 2026
659eea7
test(physical-plan): deterministic adaptive filter tests and flag-on …
adriangb Sep 7, 2026
7073c93
docs(physical-plan): tighten adaptive filter documentation
adriangb Sep 7, 2026
0adb3e5
refactor(physical-plan): evaluate the adopted order as a right-nested…
adriangb Sep 7, 2026
d2a417e
refactor(physical-plan): measure conjuncts through BinaryExpr instead…
adriangb Sep 7, 2026
173fa90
refactor(physical-plan): keep FilterExecMetrics::new signature; add w…
adriangb Sep 7, 2026
0c1933f
refactor(physical-plan): rank conjuncts by Velox's discards-per-time key
adriangb Sep 7, 2026
1f94a08
perf(physical-plan): right-nest the kept written order; pool without …
adriangb Sep 7, 2026
4262cca
refactor(physical-plan): trim adaptive filter to its essentials
adriangb Sep 7, 2026
0b8025a
docs(physical-plan): drop stale order wording and fused test name
adriangb Sep 7, 2026
be499fa
docs(physical-plan): cut comment bloat in adaptive filter
adriangb Sep 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions datafusion/common/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1137,6 +1137,17 @@ config_namespace! {
/// tables with a highly-selective join filter, but is also slightly slower.
pub enforce_batch_size_in_joins: bool, default = false

/// (experimental) When enabled, `FilterExec` measures the selectivity
/// and evaluation cost of each conjunct of an `AND` predicate at
/// runtime and reorders them to run the ones that discard the most
/// rows per unit of CPU time first. Query results never change, but
/// the observable side effects of a fallible predicate can, in either
/// direction: reordering `b <> 0 AND 1/b > 2` can make a
/// divide-by-zero error appear or disappear, since each conjunct is
/// evaluated only on the rows the conjuncts before it kept. Predicates
/// containing volatile expressions are never reordered.
pub adaptive_filter_reordering: bool, default = false

/// Size (bytes) of data buffer DataFusion uses when writing output files.
/// This affects the size of the data chunks that are uploaded to remote
/// object stores (e.g. AWS S3). If very large (>= 100 GiB) output files are being
Expand Down
Loading
Loading