Skip to content

[fix](expr opt) Handle maximum date in function rewrite - #67815

Merged
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix/date-max-boundary-rewrite
Sep 11, 2026
Merged

[fix](expr opt) Handle maximum date in function rewrite#67815
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix/date-max-boundary-rewrite

Conversation

@morrySnow

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Problem Summary:

A valid comparison against the maximum supported date failed during planning:

EXPLAIN SELECT * FROM t_datetime
WHERE DATE(dt) > '9999-12-31';

DateFunctionRewrite normally converts DATE(datetime) > date_literal into a comparison with the beginning of the following day. For 9999-12-31, constructing that bound creates year 10000, which is outside the Doris date domain and raises datetime out of range.

The rewrite now detects when the following day is outside the supported range. Because no non-null DATETIME or DATETIMEV2 value can have a date above the maximum date, it simplifies the predicate to false while preserving NULL semantics through falseOrNull. Other dates continue to use the existing range rewrite.

Unit tests cover both datetime families, nullable values, and the preceding day. Regression tests reproduce the original planning failure for DATETIME and DATETIMEV2(6) and verify both filter and projection semantics.

Release note

Fix planning failures for DATE comparisons at the maximum supported date.

Check List (For Author)

  • Test: Unit Test and Regression Test
  • Behavior changed: Yes. An impossible upper-bound predicate is safely simplified instead of failing planning.
  • Does this need documentation: No

### What problem does this PR solve?

Problem Summary:
Rewriting DATE(datetime) greater than the maximum supported date attempted to construct the following day. That produced year 10000 and aborted query planning even though the original predicate is valid and can never be true for a non-null value.

This change detects when the next day is outside the Doris date domain and simplifies the comparison to false while preserving NULL semantics. Normal dates continue to use the existing range rewrite. Tests cover legacy DATETIME, DATETIMEV2, nullable inputs, and the day immediately before the upper bound.

### Release note

Fix planning failures for DATE comparisons at the maximum supported date.

### Check List (For Author)

- Test: Unit Test and Regression Test
- Behavior changed: Yes. An impossible upper-bound predicate is safely simplified instead of failing planning.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16744 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a0a82327eb5ca6938352d65a6258dc17029e0835, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17630	3112	3087	3087
q2	2088	261	235	235
q3	10229	887	511	511
q4	4672	242	200	200
q5	7687	568	377	377
q6	134	111	93	93
q7	525	502	372	372
q8	9239	944	870	870
q9	3569	2410	2353	2353
q10	6522	858	698	698
q11	426	207	182	182
q12	625	264	194	194
q13	18168	1536	1158	1158
q14	154	150	141	141
q15	q16	447	398	365	365
q17	1392	920	794	794
q18	3028	2217	2222	2217
q19	1142	870	774	774
q20	355	281	195	195
q21	5387	1703	1846	1703
q22	318	263	225	225
Total cold run time: 93737 ms
Total hot run time: 16744 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3441	3405	3390	3390
q2	492	386	385	385
q3	2228	2289	2134	2134
q4	1172	1161	875	875
q5	2156	2092	2093	2092
q6	169	117	86	86
q7	1027	919	834	834
q8	1588	1403	1395	1395
q9	3132	3088	3063	3063
q10	1857	1790	1641	1641
q11	352	268	251	251
q12	451	432	338	338
q13	1471	1537	1176	1176
q14	178	163	157	157
q15	q16	388	397	352	352
q17	3549	3292	3211	3211
q18	4783	4382	4669	4382
q19	877	825	878	825
q20	1009	983	823	823
q21	3861	3102	3301	3102
q22	407	345	323	323
Total cold run time: 34588 ms
Total hot run time: 30835 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81822 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a0a82327eb5ca6938352d65a6258dc17029e0835, data reload: false

query5	4246	410	331	331
query6	391	137	121	121
query7	4955	396	223	223
query8	293	126	124	124
query9	8689	2869	2886	2869
query10	397	217	197	197
query11	5388	1031	899	899
query12	120	70	71	70
query13	1184	429	324	324
query14	6108	2200	2085	2085
query14_1	1981	1953	1944	1944
query15	175	126	112	112
query16	919	361	349	349
query17	783	458	389	389
query18	2328	350	244	244
query19	160	139	137	137
query20	69	70	68	68
query21	201	98	86	86
query22	5489	5395	5432	5395
query23	6473	6320	5934	5934
query23_1	6050	6062	6026	6026
query24	7258	1086	762	762
query24_1	761	746	805	746
query25	420	284	236	236
query26	1212	233	127	127
query27	2796	419	256	256
query28	4679	1461	1462	1461
query29	912	430	340	340
query30	248	150	129	129
query31	816	392	331	331
query32	128	76	67	67
query33	465	211	171	171
query34	988	839	471	471
query35	395	386	363	363
query36	544	559	541	541
query37	119	80	75	75
query38	1004	846	839	839
query39	493	497	477	477
query39_1	484	449	469	449
query40	203	88	86	86
query41	60	55	53	53
query42	74	72	71	71
query43	247	240	209	209
query44	1012	526	534	526
query45	114	112	103	103
query46	781	840	524	524
query47	753	727	696	696
query48	308	303	221	221
query49	540	250	191	191
query50	718	267	189	189
query51	8003	8021	8054	8021
query52	67	66	64	64
query53	200	207	157	157
query54	231	181	152	152
query55	68	60	56	56
query56	214	149	150	149
query57	724	745	637	637
query58	197	178	171	171
query59	1214	1212	1086	1086
query60	258	185	160	160
query61	154	132	116	116
query62	365	211	175	175
query63	172	142	139	139
query64	2856	781	661	661
query65	1597	1568	1609	1568
query66	1891	285	226	226
query67	10056	9769	9720	9720
query68	3040	1243	765	765
query69	358	218	203	203
query70	673	599	608	599
query71	263	180	169	169
query72	2376	1694	1536	1536
query73	671	592	342	342
query74	2007	1215	1135	1135
query75	1196	1108	964	964
query76	2387	714	538	538
query77	247	263	210	210
query78	4012	3590	3174	3174
query79	2748	818	587	587
query80	1652	319	271	271
query81	491	157	134	134
query82	645	136	99	99
query83	288	212	192	192
query84	296	109	92	92
query85	790	344	287	287
query86	384	172	161	161
query87	1018	972	916	916
query88	2747	2094	2118	2094
query89	281	196	174	174
query90	1961	133	131	131
query91	130	123	98	98
query92	80	63	72	63
query93	1767	1174	669	669
query94	643	247	216	216
query95	533	316	216	216
query96	837	569	269	269
query97	1046	1023	1014	1014
query98	166	136	130	130
query99	413	345	313	313
Total cold run time: 178785 ms
Total hot run time: 81822 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.58 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a0a82327eb5ca6938352d65a6258dc17029e0835, data reload: false

query1	0.01	0.00	0.00
query2	0.07	0.04	0.04
query3	0.25	0.10	0.10
query4	1.61	0.11	0.10
query5	0.19	0.16	0.17
query6	1.22	0.70	0.69
query7	0.03	0.01	0.00
query8	0.05	0.03	0.03
query9	0.29	0.22	0.21
query10	0.34	0.35	0.38
query11	0.16	0.11	0.12
query12	0.15	0.11	0.11
query13	0.31	0.32	0.30
query14	0.47	0.44	0.44
query15	0.36	0.35	0.35
query16	0.20	0.22	0.20
query17	0.70	0.67	0.70
query18	0.19	0.17	0.17
query19	1.25	1.13	1.22
query20	0.02	0.01	0.01
query21	15.44	0.15	0.11
query22	5.08	0.04	0.04
query23	16.17	0.26	0.11
query24	2.99	0.30	0.26
query25	0.12	0.03	0.03
query26	0.83	0.17	0.13
query27	0.02	0.03	0.02
query28	3.64	0.52	0.27
query29	12.50	3.15	2.57
query30	0.25	0.12	0.12
query31	2.76	0.36	0.18
query32	3.52	0.32	0.22
query33	1.36	1.42	1.37
query34	15.35	2.18	1.78
query35	1.76	1.70	1.71
query36	0.45	0.30	0.28
query37	0.06	0.04	0.04
query38	0.05	0.03	0.02
query39	0.04	0.02	0.03
query40	0.12	0.08	0.07
query41	0.07	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.53 s
Total hot run time: 14.58 s

@morrySnow morrySnow changed the title [fix](nereids) Handle maximum date in function rewrite [fix](expr opt) Handle maximum date in function rewrite Sep 11, 2026
@morrySnow
morrySnow merged commit 6172511 into apache:master Sep 11, 2026
41 of 43 checks passed
@morrySnow
morrySnow deleted the fix/date-max-boundary-rewrite branch September 11, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants