Skip to content

[fix](function) Reject even argument count of json_set/json_insert/json_replace at analysis time - #67804

Open
mrhhsg wants to merge 1 commit into
apache:masterfrom
mrhhsg:fix/json-modify-arity-check
Open

[fix](function) Reject even argument count of json_set/json_insert/json_replace at analysis time#67804
mrhhsg wants to merge 1 commit into
apache:masterfrom
mrhhsg:fix/json-modify-arity-check

Conversation

@mrhhsg

@mrhhsg mrhhsg commented Sep 10, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: None

Problem Summary:

json_set, json_insert and json_replace (and their jsonb_* aliases) take a
JSON document followed by path/value pairs, so the argument count must be odd.
The FE only checked arity >= 3, so a call whose last path had no value, e.g.
json_set('{}', '$.a', 1, '$.b'), passed analysis and EXPLAIN succeeded. The
error was only raised by the BE at execution time:

[INVALID_ARGUMENT]Function jsonb_set must have an odd number of arguments and more than 2 arguments, but got: 4

The total argument count is known at analysis time, so this PR adds
checkLegalityBeforeTypeCoercion to the three functions and rejects an even
argument count with a clear AnalysisException before type coercion and the
to_json rewrite. The BE check is kept as a defensive fallback.

Release note

None

Check List (For Author)

  • Test:
    • Unit Test: CheckExpressionLegalityTest#testJsonModifyFunctionsRejectEvenArity
    • Regression test: test_query_json_set, test_query_json_insert, test_query_json_replace
      (4/6-argument rejection including EXPLAIN and the jsonb_* aliases, legal 3/5-argument calls)
  • Behavior changed: Yes (an even argument count is now rejected during FE analysis instead of failing at BE execution)
  • Does this need documentation: No

https://claude.ai/code/session_016LqgBtqMHamfbXB2sCDnuu

…n_replace at analysis time

### What problem does this PR solve?

Issue Number: None

Problem Summary:

`json_set`, `json_insert` and `json_replace` (and their `jsonb_*` aliases) take a
JSON document followed by path/value pairs, so the argument count must be odd.
The FE only checked `arity >= 3`, so a call whose last path had no value, e.g.
`json_set('{}', '$.a', 1, '$.b')`, passed analysis and `EXPLAIN` succeeded. The
error was only raised by the BE at execution time:

```
[INVALID_ARGUMENT]Function jsonb_set must have an odd number of arguments and more than 2 arguments, but got: 4
```

The total argument count is known at analysis time, so this PR adds
`checkLegalityBeforeTypeCoercion` to the three functions and rejects an even
argument count with a clear `AnalysisException` before type coercion and the
`to_json` rewrite. The BE check is kept as a defensive fallback.

### Release note

None

### Check List (For Author)

- Test:
    - Unit Test: `CheckExpressionLegalityTest#testJsonModifyFunctionsRejectEvenArity`
    - Regression test: `test_query_json_set`, `test_query_json_insert`, `test_query_json_replace`
      (4/6-argument rejection including `EXPLAIN` and the `jsonb_*` aliases, legal 3/5-argument calls)
- Behavior changed: Yes (an even argument count is now rejected during FE analysis instead of failing at BE execution)
- Does this need documentation: No

Claude-Session: https://claude.ai/code/session_016LqgBtqMHamfbXB2sCDnuu
@mrhhsg

mrhhsg commented Sep 10, 2026

Copy link
Copy Markdown
Member Author

/review

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review result: no blocking issues found.

Critical checkpoint conclusions:

  • Goal and proof: The change rejects even total arity for json_set/json_insert/json_replace during Nereids analysis, before signature coercion and the value-to-JSON rewrite. The FE unit test covers all six canonical/alias names at invalid arities 4/6 and legal arities 3/5; regression cases cover normal queries, EXPLAIN, aliases, legal execution, and matching generated outputs.
  • Scope and clarity: The production change is a focused O(1) parity guard in the three parallel scalar classes and uses the correct Nereids AnalysisException boundary.
  • Conditions and parallel paths: The predicate matches the documented JSON-document-plus-path/value-pairs shape. All three functions and jsonb_* aliases bind to these classes; direct-bound and EXPLAIN paths use the same pre-coercion hook, and no count-changing alternate rewrite was found.
  • Compatibility: The existing FE 3-argument constructor minimum plus the new odd-arity guard exactly matches BE FunctionJsonbModify. The BE runtime guard remains in place for mixed-version or defensive fallback behavior; no protocol, symbol, or storage-format change is introduced.
  • Concurrency, lifecycle, configuration, persistence, transactions, data writes, and FE-BE variable passing: Not involved by this analysis-only change.
  • Test and result review: Negative cases use the required test/exception form; the asserted FE wording is distinct from the BE runtime error, so the oracle proves analysis-time rejection. The new single-row outputs are deterministic and consistent with insert/set/replace semantics. Tests were reviewed but not executed locally because this automated review environment prohibits builds/test runs; current GitHub CheckStyle and repository hygiene checks are passing.
  • Observability and performance: The error includes function context, actual arity, and SQL text; no additional metrics/logging are warranted. Runtime performance is unchanged, and analysis adds only a constant-time check.
  • Additional concerns: No substantiated correctness, compatibility, performance, or coverage issue remained after the main sweep and three independent Round 1 reviews.

User focus: No additional user-provided review focus was supplied; the full PR was reviewed.
Review status: Complete for head 83ae1facb891353ccbba1601be85ab5736421362; Round 1 converged with NO_NEW_VALUABLE_FINDINGS from every required subagent.

@mrhhsg
mrhhsg marked this pull request as ready for review September 11, 2026 02:02
@mrhhsg

mrhhsg commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (12/12) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16746 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 83ae1facb891353ccbba1601be85ab5736421362, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17555	3019	2986	2986
q2	2061	255	228	228
q3	10270	856	521	521
q4	4671	260	210	210
q5	7659	566	388	388
q6	136	110	91	91
q7	524	522	385	385
q8	9232	883	890	883
q9	3411	2391	2349	2349
q10	6529	841	685	685
q11	400	193	174	174
q12	616	270	208	208
q13	18127	1532	1141	1141
q14	157	149	142	142
q15	q16	437	393	368	368
q17	1366	862	819	819
q18	3114	2270	2238	2238
q19	1263	924	758	758
q20	365	283	200	200
q21	5618	1743	1924	1743
q22	327	263	229	229
Total cold run time: 93838 ms
Total hot run time: 16746 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3367	3308	3327	3308
q2	490	397	372	372
q3	2177	2308	2264	2264
q4	1172	1155	878	878
q5	2174	2125	2083	2083
q6	169	120	87	87
q7	1014	937	885	885
q8	1560	1396	1382	1382
q9	3063	3042	3059	3042
q10	1868	1816	1615	1615
q11	349	262	256	256
q12	455	436	338	338
q13	1461	1514	1160	1160
q14	167	175	155	155
q15	q16	397	395	362	362
q17	3534	3354	3198	3198
q18	4769	4386	4699	4386
q19	865	808	880	808
q20	1014	969	848	848
q21	3840	3068	3322	3068
q22	392	348	304	304
Total cold run time: 34297 ms
Total hot run time: 30799 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81575 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 83ae1facb891353ccbba1601be85ab5736421362, data reload: false

query5	4253	406	337	337
query6	391	134	123	123
query7	4956	419	232	232
query8	286	124	115	115
query9	8700	2873	2878	2873
query10	399	220	193	193
query11	5379	1029	949	949
query12	125	77	72	72
query13	1203	431	338	338
query14	6083	2204	2085	2085
query14_1	1985	1985	1965	1965
query15	177	123	109	109
query16	925	382	341	341
query17	789	451	348	348
query18	2333	326	232	232
query19	163	141	107	107
query20	73	71	72	71
query21	216	110	86	86
query22	5442	5304	5425	5304
query23	6696	6299	5989	5989
query23_1	6030	5919	5984	5919
query24	7313	1108	773	773
query24_1	787	801	767	767
query25	416	294	258	258
query26	1229	239	131	131
query27	2790	405	254	254
query28	4684	1501	1473	1473
query29	939	435	367	367
query30	251	156	133	133
query31	832	402	332	332
query32	135	75	79	75
query33	462	215	179	179
query34	979	844	489	489
query35	416	404	338	338
query36	569	616	551	551
query37	119	80	72	72
query38	1000	844	828	828
query39	489	496	473	473
query39_1	462	448	484	448
query40	203	92	78	78
query41	60	58	55	55
query42	74	72	72	72
query43	239	245	208	208
query44	993	537	535	535
query45	113	109	101	101
query46	778	818	505	505
query47	747	751	699	699
query48	307	305	232	232
query49	535	233	194	194
query50	738	261	194	194
query51	7999	7965	8010	7965
query52	74	65	58	58
query53	191	204	145	145
query54	227	160	138	138
query55	82	58	58	58
query56	202	199	158	158
query57	701	696	685	685
query58	239	168	159	159
query59	1237	1198	1106	1106
query60	243	178	183	178
query61	111	120	113	113
query62	347	216	191	191
query63	169	139	134	134
query64	2878	729	627	627
query65	1599	1583	1646	1583
query66	1843	278	209	209
query67	9804	9973	9731	9731
query68	2996	1163	717	717
query69	339	216	195	195
query70	667	632	602	602
query71	246	173	163	163
query72	2239	1657	1505	1505
query73	625	598	330	330
query74	1992	1219	1133	1133
query75	1163	1092	950	950
query76	2385	720	504	504
query77	250	250	215	215
query78	3999	3747	3211	3211
query79	2741	806	565	565
query80	1645	318	275	275
query81	517	155	134	134
query82	787	124	100	100
query83	275	209	184	184
query84	291	111	90	90
query85	812	338	282	282
query86	475	174	173	173
query87	1014	967	896	896
query88	3059	2103	2098	2098
query89	266	193	175	175
query90	2177	126	117	117
query91	128	119	91	91
query92	92	68	69	68
query93	2380	1033	655	655
query94	624	248	222	222
query95	517	251	237	237
query96	844	585	265	265
query97	1043	1068	1010	1010
query98	176	136	130	130
query99	414	332	309	309
Total cold run time: 179824 ms
Total hot run time: 81575 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 83ae1facb891353ccbba1601be85ab5736421362, data reload: false

query1	0.01	0.01	0.00
query2	0.08	0.04	0.04
query3	0.25	0.11	0.10
query4	1.60	0.10	0.11
query5	0.17	0.16	0.16
query6	1.27	0.72	0.67
query7	0.02	0.00	0.00
query8	0.04	0.02	0.03
query9	0.28	0.20	0.21
query10	0.35	0.36	0.34
query11	0.16	0.11	0.12
query12	0.14	0.11	0.12
query13	0.30	0.30	0.31
query14	0.44	0.43	0.45
query15	0.36	0.35	0.35
query16	0.21	0.22	0.22
query17	0.72	0.68	0.71
query18	0.18	0.15	0.16
query19	1.16	1.15	1.17
query20	0.02	0.01	0.01
query21	15.44	0.16	0.11
query22	5.07	0.04	0.04
query23	16.16	0.26	0.10
query24	3.00	0.32	0.26
query25	0.11	0.03	0.03
query26	0.81	0.16	0.12
query27	0.04	0.03	0.03
query28	3.64	0.54	0.25
query29	12.49	3.19	2.55
query30	0.26	0.11	0.13
query31	2.76	0.38	0.17
query32	3.52	0.33	0.23
query33	1.39	1.49	1.41
query34	15.37	2.18	1.78
query35	1.74	1.72	1.73
query36	0.48	0.28	0.29
query37	0.06	0.04	0.04
query38	0.04	0.03	0.02
query39	0.03	0.03	0.02
query40	0.11	0.08	0.07
query41	0.08	0.02	0.02
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 90.42 s
Total hot run time: 14.56 s

@morrySnow morrySnow changed the title [fix](nereids) Reject even argument count of json_set/json_insert/json_replace at analysis time [fix](function) Reject even argument count of json_set/json_insert/json_replace at analysis time Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants