Skip to content

Conversation

flashzxi
Copy link

@flashzxi flashzxi commented Sep 5, 2025

fix FilteredRules::filterValidRules

issue: #55718

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@flashzxi
Copy link
Author

flashzxi commented Sep 5, 2025

    @Override
    public List<Rule> filterValidRules(CascadesContext cascadesContext) {
        BitSet disableRules = cascadesContext.getAndCacheDisableRules();
        if (disableRules.isEmpty()) {
            return allRules;
        }
        List<Rule> validRules = new ArrayList<>(allRules);
        for (Rule rule : allRules) {
            if (!disableRules.get(rule.getRuleType().type())) {
                validRules.add(rule);
            }
        }
        return validRules;
    }

will filter nothing and add repeat item.

List<Rule> validRules = new ArrayList<>(allRules);->
List<Rule> validRules = new ArrayList<>();
to fix this problem

@flashzxi flashzxi changed the title fix FilteredRules::filterValidRules fix FilteredRules::filterValidRules will filter nothing Sep 5, 2025
@flashzxi flashzxi changed the title fix FilteredRules::filterValidRules will filter nothing [Fix](Cascades) FilteredRules::filterValidRules will filter nothing Sep 5, 2025
@flashzxi
Copy link
Author

flashzxi commented Sep 5, 2025

run buildall

@flashzxi flashzxi changed the title [Fix](Cascades) FilteredRules::filterValidRules will filter nothing [fix](cascades) FilteredRules::filterValidRules will filter nothing Sep 5, 2025
@flashzxi
Copy link
Author

flashzxi commented Sep 5, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34010 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 26dbafadbe75dbf2abd688fd39881ccd362dcf05, data reload: false

------ Round 1 ----------------------------------
q1	17639	5260	5044	5044
q2	2020	313	205	205
q3	10321	1306	732	732
q4	10224	1022	539	539
q5	7529	2417	2393	2393
q6	182	173	137	137
q7	946	764	667	667
q8	9360	1359	1122	1122
q9	6910	5104	5214	5104
q10	6901	2398	1967	1967
q11	489	306	287	287
q12	354	357	231	231
q13	17774	3629	2997	2997
q14	234	248	219	219
q15	554	510	491	491
q16	428	431	381	381
q17	589	876	371	371
q18	7454	7121	7040	7040
q19	1241	964	550	550
q20	348	348	230	230
q21	3908	2546	2317	2317
q22	1063	1023	986	986
Total cold run time: 106468 ms
Total hot run time: 34010 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5094	5087	5064	5064
q2	267	338	239	239
q3	2210	2672	2317	2317
q4	1350	1777	1349	1349
q5	4239	4228	4650	4228
q6	225	181	139	139
q7	2070	1953	1889	1889
q8	2776	2713	2748	2713
q9	7499	7358	7349	7349
q10	3084	3273	2852	2852
q11	588	541	525	525
q12	713	775	682	682
q13	3430	4018	3332	3332
q14	300	306	282	282
q15	542	506	496	496
q16	455	541	488	488
q17	1230	1586	1419	1419
q18	7834	7663	7617	7617
q19	848	824	849	824
q20	2014	2076	1921	1921
q21	5067	4657	4393	4393
q22	1084	1024	1009	1009
Total cold run time: 52919 ms
Total hot run time: 51127 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186856 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 26dbafadbe75dbf2abd688fd39881ccd362dcf05, data reload: false

query1	1093	463	413	413
query2	6574	1783	1706	1706
query3	6758	236	224	224
query4	25877	23565	22933	22933
query5	4440	680	528	528
query6	340	262	245	245
query7	4657	526	310	310
query8	300	262	265	262
query9	8666	2988	2957	2957
query10	523	365	313	313
query11	15688	15057	14732	14732
query12	195	129	130	129
query13	1687	574	442	442
query14	9554	6048	5972	5972
query15	213	204	175	175
query16	7692	683	502	502
query17	1265	780	655	655
query18	2060	449	394	394
query19	208	197	177	177
query20	136	121	119	119
query21	217	129	118	118
query22	4143	4310	4018	4018
query23	33869	33000	33209	33000
query24	8239	2365	2407	2365
query25	561	519	454	454
query26	1238	276	170	170
query27	2682	507	361	361
query28	4380	2278	2229	2229
query29	783	613	505	505
query30	296	231	203	203
query31	917	850	749	749
query32	88	86	81	81
query33	589	398	369	369
query34	785	866	528	528
query35	840	824	758	758
query36	976	1020	897	897
query37	132	113	93	93
query38	4187	4037	3982	3982
query39	1496	1497	1429	1429
query40	235	137	135	135
query41	66	66	61	61
query42	133	123	119	119
query43	561	517	472	472
query44	1357	883	874	874
query45	188	190	171	171
query46	862	1020	657	657
query47	1767	1783	1734	1734
query48	389	425	336	336
query49	759	506	423	423
query50	646	706	406	406
query51	4204	4266	4256	4256
query52	121	116	106	106
query53	257	260	205	205
query54	617	606	548	548
query55	95	99	96	96
query56	347	328	323	323
query57	1178	1234	1130	1130
query58	290	275	286	275
query59	2577	2688	2619	2619
query60	358	355	347	347
query61	176	160	156	156
query62	857	741	665	665
query63	240	206	203	203
query64	4404	1157	828	828
query65	4301	4177	4210	4177
query66	1088	443	350	350
query67	15541	15253	15228	15228
query68	9335	926	602	602
query69	492	342	302	302
query70	1272	1129	1136	1129
query71	603	353	325	325
query72	6037	5001	5094	5001
query73	751	670	382	382
query74	8883	8983	8719	8719
query75	4332	3097	2663	2663
query76	3731	1149	749	749
query77	851	428	343	343
query78	9455	9727	8765	8765
query79	2712	843	613	613
query80	694	572	521	521
query81	466	329	225	225
query82	453	141	125	125
query83	295	269	246	246
query84	305	112	101	101
query85	878	466	430	430
query86	377	322	326	322
query87	4314	4199	4185	4185
query88	2924	2270	2249	2249
query89	416	332	301	301
query90	1962	229	233	229
query91	162	164	134	134
query92	93	80	69	69
query93	1548	1008	662	662
query94	694	397	324	324
query95	410	342	337	337
query96	492	603	286	286
query97	2686	2664	2582	2582
query98	241	220	227	220
query99	1445	1422	1318	1318
Total cold run time: 277267 ms
Total hot run time: 186856 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26dbafadbe75dbf2abd688fd39881ccd362dcf05, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.06	0.05
query3	0.26	0.09	0.09
query4	1.62	0.12	0.12
query5	0.46	0.43	0.41
query6	1.17	0.65	0.65
query7	0.04	0.03	0.02
query8	0.05	0.04	0.05
query9	0.60	0.53	0.52
query10	0.58	0.58	0.58
query11	0.17	0.12	0.12
query12	0.15	0.12	0.13
query13	0.64	0.63	0.62
query14	0.79	0.84	0.85
query15	0.86	0.86	0.86
query16	0.38	0.41	0.39
query17	1.08	1.04	1.05
query18	0.22	0.20	0.20
query19	2.02	1.85	1.77
query20	0.02	0.01	0.01
query21	15.40	0.98	0.58
query22	0.80	1.15	0.95
query23	14.73	1.41	0.60
query24	6.84	1.67	0.60
query25	0.45	0.24	0.09
query26	0.64	0.16	0.13
query27	0.06	0.06	0.05
query28	9.36	0.91	0.42
query29	12.56	3.83	3.22
query30	0.28	0.14	0.11
query31	2.82	0.59	0.39
query32	3.23	0.56	0.46
query33	3.10	3.15	3.13
query34	15.87	5.47	4.81
query35	4.90	4.90	4.91
query36	0.71	0.51	0.50
query37	0.11	0.08	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 103.56 s
Total hot run time: 29.81 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants