fix: reject invalid placeholders in CREATE FUNCTION bodies at definition time - #25039
Open
quwin wants to merge 4 commits into
Open
fix: reject invalid placeholders in CREATE FUNCTION bodies at definition time#25039quwin wants to merge 4 commits into
quwin wants to merge 4 commits into
Conversation
…ion time Positional placeholders in a SQL-function RETURN body that do not reference a declared argument (e.g. `$3` for a function declared with two arguments) were accepted at CREATE FUNCTION and only failed when the function was invoked. Validate them in the SQL planner's CreateFunction arm, where both the declared argument list and the parsed body are available, so invalid definitions are rejected for every FunctionFactory without changing PREPARE's permissive parameter inference. Generated with Codebuff 🤖 Co-Authored-By: Codebuff <noreply@codebuff.com>
…ation CREATE FUNCTION bodies with invalid placeholders now fail during planning (apache#25038), so the sqllogictest case that previously used an out-of-range placeholder to reach the "function factory has not been configured" error now uses a valid body instead, and the new definition-time errors are covered explicitly. Generated with Codebuff 🤖 Co-Authored-By: Codebuff <noreply@codebuff.com>
Author
|
Hi, this is my first PR to DataFusion. Could a committer please run the CI checks? Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
A
CREATE FUNCTIONwith a SQL body is accepted even when itsRETURNexpression references a positional placeholder that does not match a declared argument:The invalid definition is registered successfully, and the error only surfaces when the function is invoked:
A definition error like this should be reported at
CREATE FUNCTIONtime, not deferred to every invocation. This is acknowledged in code by two FIXMEs (see #25038):datafusion/sql/src/expr/value.rs— "In the CREATE FUNCTION branch, param_type = None should raise an error"datafusion/core/tests/user_defined/user_defined_scalar_functions.rs— "Definitions with invalid placeholders are allowed, fail at runtime"What changes are included in this PR?
Statement::CreateFunctionarm of the SQL planner (datafusion/sql/src/statement.rs): after theRETURNbody is planned, walk it forExpr::Placeholdernodes and reject any positional$Noutside1..=declared_arg_countwithInvalid placeholder, out of range: $N, and any named placeholder (which can only survive parsing when zero arguments were declared) withUnknown placeholder: $N.value.rsor in the factories) means it applies to everyFunctionFactoryimplementation, and it cannot changePREPAREsemantics, where an empty/unknown parameter list must stay permissive for deferred type inference. Runtime guards in the factories remain as a defensive backstop.datafusion/sql/src/expr/value.rs.create_function.slt: the case that previously used an out-of-range placeholder ($1 + $2with one declared argument) only to reach the "function factory has not been configured" error now uses a valid body (RETURN $1), and the new definition-time errors are asserted explicitly (out-of-range$2; zero-argument function referencing$1).What is the testing strategy for this PR?
create_scalar_function_from_sql_statement(): the invalidCREATE FUNCTIONmust now fail withError during planning: Invalid placeholder, out of range: $3.create_scalar_function_from_sql_statement_invalid_placeholders()covering: out-of-range positional with positional declared args; out-of-range positional with named declared args (RETURN $a + $3); a zero-argument function referencing$1; a zero-argument function referencing a named$a; and a positive case (RETURN $1 + $2) that must still be accepted.create_function.sltas described above.cargo test -p datafusion --test user_defined_integration,cargo test -p datafusion-sql,cargo test -p datafusion-sqllogictest create_function(andprepare),cargo fmt --check, and clippy (-D warnings) on the touched crates are all green.