List view

Is it possible to support PyArrow backed UDFs in Comet natively?

enhancementNew feature or requestNew feature or request

Status: Open.

#957 In apache/datafusion-comet;

· SemyonSinchenko opened on Sep 21, 2024

Register CometArrowAllocator as a Spark MemoryConsumer for JVM-UDF dispatch

area:expressionsExpression evaluationExpression evaluation

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4174 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Eliminate remaining row<->Arrow round-trip in CometPythonMapInArrowExec

area:udf

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4240 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

Add native implementations ofRegExpExtract and RegExpExtractAll

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

good first issueGood for newcomersGood for newcomers

Status: Open.

#2708 In apache/datafusion-comet;

· andygrove opened on Nov 6, 2025

Re-enable COUNT for mixed Spark partial / Comet final aggregate execution

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4242 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

[hdfs-opendal] Got an reader got too little data error on hdfs-opendal

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#2400 In apache/datafusion-comet;

· wForget opened on Sep 16, 2025

ExecutionMemoryPool errors releasing more memory than allocated

bugSomething isn't workingSomething isn't working

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

Status: Open.

#2453 In apache/datafusion-comet;

· andygrove opened on Sep 25, 2025

Dropping a GlobalRef in a detached thread

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

bugSomething isn't workingSomething isn't working

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

Status: Open.

#2470 In apache/datafusion-comet;

· andygrove opened on Sep 26, 2025

Enable windowed aggregates and fix correctness issues

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

correctness

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4007 In apache/datafusion-comet;

· andygrove opened on Apr 20, 2026

feat: Add 100% Spark-compatible regex support via codegen dispatcher

Status: Open (in progress).

apache/datafusion-cometnumber 4239

#4239 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

[native_datafusion] Add support for reading row index metadata columns

enhancementNew feature or requestNew feature or request

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

Status: Open.

#3432 In apache/datafusion-comet;

· andygrove opened on Feb 6, 2026

chore: DataFusion 54.0.0

buildBuild environmentBuild environment

dependenciesPull requests that update a dependency filePull requests that update a dependency file

enhancementNew feature or requestNew feature or request

Status: Open.

#3978 In apache/datafusion-comet;

· comphead opened on Apr 17, 2026

Implement JVM UDFs for all date/time expressions

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4311 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

feat: 100% Spark-compatible JSON support via codegen dispatcher

Status: Draft (not ready).

apache/datafusion-cometnumber 4305

#4305 In apache/datafusion-comet;

· andygrove opened on May 12, 2026

Implement JVM UDFs for JSON expressions

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4313 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

native_datafusion more permissive than Spark 3.x when reading Parquet TimestampNTZ columns

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

spark 3.x

Status: Open.

#4219 In apache/datafusion-comet;

· andygrove opened on May 5, 2026

native_datafusion: ParquetSchemaConvert error does not include the file path

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

Status: Open.

#4316 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

FIRST/LAST returns wrong result with PartialMerge

bugSomething isn't workingSomething isn't working

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4131 In apache/datafusion-comet;

· comphead opened on Apr 28, 2026

Establish nomenclature style guide and audit operator names for clarity

documentationImprovements or additions to documentationImprovements or additions to documentation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4419 In apache/datafusion-comet;

· andygrove opened on May 24, 2026

Add a docs-review Claude skill to enforce style guide and documentation quality

documentationImprovements or additions to documentationImprovements or additions to documentation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4420 In apache/datafusion-comet;

· andygrove opened on May 24, 2026

Reorganize user and contributor guide navigation for clearer information architecture

documentationImprovements or additions to documentationImprovements or additions to documentation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4421 In apache/datafusion-comet;

· andygrove opened on May 24, 2026

chore: audit date/time expressions

Status: Open (in progress).

apache/datafusion-cometnumber 4448

#4448 In apache/datafusion-comet;

· andygrove opened on May 27, 2026

chore(audit): audit string expressions across Spark 3.4.3, 3.5.8, 4.0.1

Status: Open (in progress).

apache/datafusion-cometnumber 4461

#4461 In apache/datafusion-comet;

· andygrove opened on May 27, 2026

Add distinction between "info" and "fallback" messages

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4006 In apache/datafusion-comet;

· andygrove opened on Apr 20, 2026

Math expression audit follow-ups (from #4486)

area:expressionsExpression evaluationExpression evaluation

correctness

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4500 In apache/datafusion-comet;

· andygrove opened on May 28, 2026