Skip to content

[fix](aggregate) Restrict sum literal reassociation to integers - #67801

Merged
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix/sum-literal-integer-only
Sep 14, 2026
Merged

morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix/sum-literal-integer-only

Conversation

@morrySnow

Copy link
Copy Markdown
Contributor

Problem

When a query contains multiple aggregates such as SUM(x + 1.0) and SUM(x + 2.0), the optimizer reassociates them into SUM(x) plus COUNT(x) times each literal. For floating-point inputs this changes IEEE-754 rounding from per-row addition to post-aggregation addition and can return a different result.

Root cause

SumLiteralRewrite accepted both integer and floating-point literals, but did not require the analyzed input expression and arithmetic result to be integer types. The transformation is not semantics-preserving for floating-point arithmetic.

Reproduction

With DOUBLE values 1e16 and -1e16, the two original aggregates produce 0 and 4. Before this change, the rewritten plan produced 2 and 4 and exposed COUNT in the aggregate plan.

Fix

Restrict the reassociation to expressions whose analyzed child, non-literal operand, and literal operand are all integer-like. Floating-point and decimal expressions remain unchanged. Existing widening integer-cast handling is preserved. Integer overflow behavior is intentionally unchanged.

Tests

  • Added unit coverage for integer, float, double, decimal, and mixed operand types.
  • Added a regression case for the floating-point rounding reproduction and verified that its plan is not rewritten.
  • Verified that integer aggregates still use the optimization.
  • Ran the focused FE unit test, the full FE build with checkstyle, and the sumRewrite regression suite.

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 33.33% (1/3) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16783 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 459755834752947961255f7a3bc1beb45e8e9ec9, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17589	3117	3094	3094
q2	2062	269	222	222
q3	10275	894	538	538
q4	4668	246	210	210
q5	7678	603	385	385
q6	137	114	97	97
q7	517	505	381	381
q8	9238	900	893	893
q9	3409	2401	2399	2399
q10	6540	847	712	712
q11	399	199	176	176
q12	620	255	196	196
q13	18121	1537	1142	1142
q14	159	143	141	141
q15	q16	439	402	374	374
q17	1367	844	796	796
q18	3100	2244	2221	2221
q19	1262	945	782	782
q20	373	290	212	212
q21	5683	1583	1808	1583
q22	328	274	229	229
Total cold run time: 93964 ms
Total hot run time: 16783 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3456	3400	3396	3396
q2	494	395	373	373
q3	2234	2250	2156	2156
q4	1202	1160	897	897
q5	2175	2146	2064	2064
q6	164	115	87	87
q7	1012	912	837	837
q8	1575	1403	1396	1396
q9	3085	3053	3045	3045
q10	1874	1816	1634	1634
q11	348	269	248	248
q12	448	423	342	342
q13	1483	1526	1148	1148
q14	173	162	155	155
q15	q16	390	395	361	361
q17	3521	3242	3135	3135
q18	4774	4342	4741	4342
q19	840	810	887	810
q20	1032	961	845	845
q21	3851	3112	3271	3112
q22	409	341	318	318
Total cold run time: 34540 ms
Total hot run time: 30701 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81273 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 459755834752947961255f7a3bc1beb45e8e9ec9, data reload: false

query5	4237	418	330	330
query6	379	148	124	124
query7	4950	418	235	235
query8	291	129	120	120
query9	8702	2868	2880	2868
query10	399	216	173	173
query11	5394	1059	927	927
query12	122	74	70	70
query13	1200	453	338	338
query14	6009	2201	2085	2085
query14_1	1968	1978	1984	1978
query15	180	123	112	112
query16	926	372	367	367
query17	791	436	333	333
query18	2317	321	231	231
query19	157	135	103	103
query20	70	70	66	66
query21	194	100	87	87
query22	5412	5405	5356	5356
query23	6835	6257	5879	5879
query23_1	6142	5956	6146	5956
query24	7244	1082	754	754
query24_1	781	735	785	735
query25	401	278	226	226
query26	1222	239	128	128
query27	2779	412	256	256
query28	4685	1492	1512	1492
query29	907	409	328	328
query30	243	154	134	134
query31	815	394	333	333
query32	133	72	72	72
query33	452	198	173	173
query34	993	821	478	478
query35	414	397	340	340
query36	588	566	526	526
query37	120	82	73	73
query38	999	850	834	834
query39	483	510	465	465
query39_1	468	460	441	441
query40	212	93	83	83
query41	59	57	56	56
query42	75	77	72	72
query43	239	240	210	210
query44	1006	546	549	546
query45	113	108	104	104
query46	779	859	511	511
query47	762	757	724	724
query48	320	309	241	241
query49	556	263	200	200
query50	725	264	201	201
query51	8105	7925	7897	7897
query52	72	72	63	63
query53	195	207	153	153
query54	248	178	149	149
query55	81	61	56	56
query56	210	178	177	177
query57	694	624	664	624
query58	231	212	167	167
query59	1220	1270	1101	1101
query60	233	180	170	170
query61	109	115	106	106
query62	378	202	180	180
query63	174	154	145	145
query64	2676	729	595	595
query65	1662	1617	1600	1600
query66	1895	277	200	200
query67	9572	9604	9691	9604
query68	2988	1225	744	744
query69	348	243	202	202
query70	662	614	616	614
query71	255	175	153	153
query72	2287	1660	1464	1464
query73	648	603	338	338
query74	1982	1224	1127	1127
query75	1197	1090	950	950
query76	2346	705	527	527
query77	244	257	205	205
query78	3907	3652	3201	3201
query79	2328	788	562	562
query80	1613	325	255	255
query81	491	155	132	132
query82	818	134	100	100
query83	278	209	186	186
query84	301	112	87	87
query85	790	371	280	280
query86	393	173	175	173
query87	1029	969	889	889
query88	2775	2093	2116	2093
query89	289	196	175	175
query90	1974	132	129	129
query91	128	116	98	98
query92	79	69	72	69
query93	1551	1111	701	701
query94	642	226	213	213
query95	506	319	227	227
query96	838	596	262	262
query97	1078	1037	1013	1013
query98	162	136	132	132
query99	419	342	308	308
Total cold run time: 177772 ms
Total hot run time: 81273 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.63 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 459755834752947961255f7a3bc1beb45e8e9ec9, data reload: false

query1	0.01	0.01	0.01
query2	0.07	0.04	0.03
query3	0.26	0.11	0.11
query4	1.60	0.09	0.10
query5	0.17	0.17	0.16
query6	1.33	0.70	0.69
query7	0.03	0.00	0.00
query8	0.04	0.03	0.03
query9	0.30	0.22	0.22
query10	0.34	0.34	0.34
query11	0.16	0.12	0.12
query12	0.14	0.12	0.11
query13	0.31	0.31	0.30
query14	0.46	0.47	0.46
query15	0.37	0.35	0.34
query16	0.22	0.20	0.22
query17	0.72	0.64	0.66
query18	0.18	0.17	0.17
query19	1.19	1.15	1.14
query20	0.02	0.01	0.01
query21	15.44	0.16	0.13
query22	5.04	0.04	0.05
query23	16.17	0.26	0.10
query24	3.02	0.30	0.24
query25	0.11	0.04	0.03
query26	0.80	0.17	0.12
query27	0.03	0.02	0.02
query28	3.65	0.58	0.27
query29	12.44	3.31	2.65
query30	0.28	0.11	0.11
query31	2.76	0.38	0.17
query32	3.52	0.32	0.24
query33	1.51	1.36	1.51
query34	15.40	2.24	1.78
query35	1.74	1.75	1.70
query36	0.45	0.31	0.28
query37	0.06	0.04	0.04
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.12	0.08	0.07
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.03	0.02	0.03
Total cold run time: 90.68 s
Total hot run time: 14.63 s

@morrySnow morrySnow changed the title [fix](nereids) Restrict sum literal reassociation to integers [fix](aggregate) Restrict sum literal reassociation to integers Sep 11, 2026
@morrySnow
morrySnow merged commit 08fd5f8 into apache:master Sep 14, 2026
40 of 42 checks passed
@morrySnow
morrySnow deleted the fix/sum-literal-integer-only branch September 14, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants