Skip to content

fix: reject invalid placeholders in CREATE FUNCTION bodies at definition time - #25039

Open
quwin wants to merge 4 commits into
apache:mainfrom
quwin:fix/udf-invalid-placeholder-validation
Open

fix: reject invalid placeholders in CREATE FUNCTION bodies at definition time#25039
quwin wants to merge 4 commits into
apache:mainfrom
quwin:fix/udf-invalid-placeholder-validation

Conversation

@quwin

@quwin quwin commented Sep 7, 2026

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

A CREATE FUNCTION with a SQL body is accepted even when its RETURN expression references a positional placeholder that does not match a declared argument:

CREATE FUNCTION better_add(DOUBLE, DOUBLE)
    RETURNS DOUBLE
    RETURN $1 + $3   -- only two arguments are declared

The invalid definition is registered successfully, and the error only surfaces when the function is invoked:

Optimizer rule 'simplify_expressions' failed
caused by
Execution error: Invalid placeholder, out of range: $3

A definition error like this should be reported at CREATE FUNCTION time, not deferred to every invocation. This is acknowledged in code by two FIXMEs (see #25038):

  • datafusion/sql/src/expr/value.rs — "In the CREATE FUNCTION branch, param_type = None should raise an error"
  • datafusion/core/tests/user_defined/user_defined_scalar_functions.rs — "Definitions with invalid placeholders are allowed, fail at runtime"

What changes are included in this PR?

  • Validate placeholders at planning time in the Statement::CreateFunction arm of the SQL planner (datafusion/sql/src/statement.rs): after the RETURN body is planned, walk it for Expr::Placeholder nodes and reject any positional $N outside 1..=declared_arg_count with Invalid placeholder, out of range: $N, and any named placeholder (which can only survive parsing when zero arguments were declared) with Unknown placeholder: $N.
  • The check uses the declared argument count, so functions with defaulted arguments remain callable with fewer arguments than declared.
  • Doing this in the planner (rather than in value.rs or in the factories) means it applies to every FunctionFactory implementation, and it cannot change PREPARE semantics, where an empty/unknown parameter list must stay permissive for deferred type inference. Runtime guards in the factories remain as a defensive backstop.
  • Replaced the now-superseded FIXME comment in datafusion/sql/src/expr/value.rs.
  • Updated create_function.slt: the case that previously used an out-of-range placeholder ($1 + $2 with one declared argument) only to reach the "function factory has not been configured" error now uses a valid body (RETURN $1), and the new definition-time errors are asserted explicitly (out-of-range $2; zero-argument function referencing $1).

What is the testing strategy for this PR?

  • Flipped the FIXME regression test in create_scalar_function_from_sql_statement(): the invalid CREATE FUNCTION must now fail with Error during planning: Invalid placeholder, out of range: $3.
  • Added create_scalar_function_from_sql_statement_invalid_placeholders() covering: out-of-range positional with positional declared args; out-of-range positional with named declared args (RETURN $a + $3); a zero-argument function referencing $1; a zero-argument function referencing a named $a; and a positive case (RETURN $1 + $2) that must still be accepted.
  • Extended create_function.slt as described above.
  • Verified: pre-fix the new tests fail (CREATE succeeded), post-fix all pass. cargo test -p datafusion --test user_defined_integration, cargo test -p datafusion-sql, cargo test -p datafusion-sqllogictest create_function (and prepare), cargo fmt --check, and clippy (-D warnings) on the touched crates are all green.

quwin and others added 2 commits September 7, 2026 09:57
…ion time

Positional placeholders in a SQL-function RETURN body that do not
reference a declared argument (e.g. `$3` for a function declared with
two arguments) were accepted at CREATE FUNCTION and only failed when the
function was invoked. Validate them in the SQL planner's CreateFunction
arm, where both the declared argument list and the parsed body are
available, so invalid definitions are rejected for every FunctionFactory
without changing PREPARE's permissive parameter inference.

Generated with Codebuff 🤖
Co-Authored-By: Codebuff <noreply@codebuff.com>
@github-actions github-actions Bot added sql SQL Planner core Core DataFusion crate labels Sep 7, 2026
…ation

CREATE FUNCTION bodies with invalid placeholders now fail during planning
(apache#25038), so the sqllogictest case that previously used an out-of-range
placeholder to reach the "function factory has not been configured" error
now uses a valid body instead, and the new definition-time errors are
covered explicitly.

Generated with Codebuff 🤖
Co-Authored-By: Codebuff <noreply@codebuff.com>
@github-actions github-actions Bot added the sqllogictest SQL Logic Tests (.slt) label Sep 7, 2026
@quwin

quwin commented Sep 7, 2026

Copy link
Copy Markdown
Author

Hi, this is my first PR to DataFusion. Could a committer please run the CI checks? Thanks!
@jayzhan211

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CREATE FUNCTION accepts placeholders that don't match a declared argument; the error is deferred to call time

1 participant