Skip to content

Conversation

airborne12
Copy link
Member

@airborne12 airborne12 commented Sep 5, 2025

This reverts commit #47841

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #47841

Problem Summary:
This optimization for key column when using count agg may cause wrong result, like select COUNT(nullable_pk)

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Sep 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34194 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 74b5e69117f810b5d0df3169c5959ae1ea3296fc, data reload: false

------ Round 1 ----------------------------------
q1	17636	5203	5056	5056
q2	2030	316	220	220
q3	10266	1331	726	726
q4	10231	1023	531	531
q5	7558	2398	2438	2398
q6	184	172	137	137
q7	939	782	621	621
q8	9346	1339	1111	1111
q9	7062	5117	5180	5117
q10	6897	2380	1978	1978
q11	474	316	275	275
q12	350	378	235	235
q13	17759	3688	3042	3042
q14	245	231	235	231
q15	566	500	485	485
q16	454	447	382	382
q17	600	871	367	367
q18	7443	7177	7172	7172
q19	1102	973	584	584
q20	363	353	235	235
q21	3855	3235	2334	2334
q22	1090	1029	957	957
Total cold run time: 106450 ms
Total hot run time: 34194 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5133	5114	5060	5060
q2	263	344	235	235
q3	2215	2711	2315	2315
q4	1327	1772	1326	1326
q5	4218	4353	4609	4353
q6	225	176	138	138
q7	2026	1983	1814	1814
q8	2719	2676	2686	2676
q9	7385	7689	7408	7408
q10	3108	3324	2888	2888
q11	588	523	519	519
q12	687	797	665	665
q13	3488	3904	3400	3400
q14	280	303	289	289
q15	545	495	492	492
q16	433	472	462	462
q17	1227	1664	1360	1360
q18	7899	7650	7594	7594
q19	815	850	862	850
q20	2039	2151	1948	1948
q21	4930	4640	4449	4449
q22	1072	1080	999	999
Total cold run time: 52622 ms
Total hot run time: 51240 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187183 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 74b5e69117f810b5d0df3169c5959ae1ea3296fc, data reload: false

query1	1072	421	414	414
query2	6567	1707	1716	1707
query3	6749	227	225	225
query4	26125	23549	23609	23549
query5	4530	673	533	533
query6	347	257	246	246
query7	4695	517	298	298
query8	299	247	239	239
query9	8636	2900	2883	2883
query10	470	359	316	316
query11	15694	15001	15188	15001
query12	174	122	117	117
query13	1679	551	451	451
query14	9524	5812	5801	5801
query15	218	201	188	188
query16	7547	667	494	494
query17	1234	766	651	651
query18	2056	440	340	340
query19	215	212	183	183
query20	136	127	126	126
query21	219	131	126	126
query22	4458	4327	4026	4026
query23	33869	33181	32865	32865
query24	8224	2416	2456	2416
query25	561	511	454	454
query26	1250	280	166	166
query27	2717	517	359	359
query28	4378	2237	2217	2217
query29	787	625	571	571
query30	293	225	207	207
query31	909	814	712	712
query32	96	86	79	79
query33	587	402	367	367
query34	800	866	524	524
query35	828	835	749	749
query36	959	1044	914	914
query37	128	115	95	95
query38	4059	4021	3994	3994
query39	1488	1458	1426	1426
query40	231	133	131	131
query41	67	85	62	62
query42	129	116	117	116
query43	500	526	455	455
query44	1347	867	861	861
query45	184	175	171	171
query46	866	1031	653	653
query47	1774	1827	1740	1740
query48	413	424	327	327
query49	761	503	457	457
query50	639	706	399	399
query51	4157	4162	4110	4110
query52	121	114	114	114
query53	243	274	205	205
query54	621	606	560	560
query55	92	90	89	89
query56	354	360	320	320
query57	1205	1197	1139	1139
query58	287	281	283	281
query59	2628	2674	2621	2621
query60	365	358	348	348
query61	164	159	161	159
query62	822	751	657	657
query63	233	197	200	197
query64	4479	1140	835	835
query65	4303	4182	4220	4182
query66	1188	487	361	361
query67	15536	15553	14996	14996
query68	9753	939	593	593
query69	488	335	294	294
query70	1273	1178	1116	1116
query71	608	360	337	337
query72	5746	5021	5184	5021
query73	758	690	368	368
query74	9190	8830	8882	8830
query75	4368	3087	2642	2642
query76	4031	1205	742	742
query77	1045	424	331	331
query78	9584	9905	8803	8803
query79	1193	843	613	613
query80	634	585	532	532
query81	473	261	232	232
query82	424	144	115	115
query83	286	278	314	278
query84	264	116	95	95
query85	879	475	434	434
query86	347	329	311	311
query87	4329	4264	4237	4237
query88	2828	2263	2262	2262
query89	404	333	305	305
query90	1938	232	226	226
query91	168	172	139	139
query92	96	77	73	73
query93	1083	1040	658	658
query94	699	427	332	332
query95	409	331	337	331
query96	486	608	287	287
query97	2682	2693	2573	2573
query98	248	223	219	219
query99	1462	1436	1279	1279
Total cold run time: 276462 ms
Total hot run time: 187183 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.52 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 74b5e69117f810b5d0df3169c5959ae1ea3296fc, data reload: false

query1	0.06	0.06	0.05
query2	0.09	0.05	0.05
query3	0.25	0.09	0.08
query4	1.61	0.11	0.12
query5	0.45	0.44	0.42
query6	1.18	0.64	0.66
query7	0.03	0.02	0.02
query8	0.05	0.05	0.05
query9	0.61	0.54	0.53
query10	0.58	0.58	0.58
query11	0.17	0.12	0.12
query12	0.15	0.12	0.12
query13	0.62	0.62	0.63
query14	0.79	0.84	0.86
query15	0.89	0.86	0.86
query16	0.39	0.43	0.39
query17	1.06	1.05	1.02
query18	0.22	0.20	0.20
query19	1.98	1.87	1.84
query20	0.01	0.01	0.01
query21	15.40	0.92	0.59
query22	0.75	1.23	0.81
query23	14.73	1.33	0.62
query24	7.28	0.93	0.31
query25	0.30	0.21	0.09
query26	0.61	0.17	0.15
query27	0.06	0.05	0.05
query28	9.17	0.90	0.42
query29	12.60	3.91	3.24
query30	0.29	0.13	0.11
query31	2.83	0.62	0.38
query32	3.24	0.57	0.48
query33	3.19	3.12	3.07
query34	16.01	5.57	4.86
query35	4.93	4.94	4.92
query36	0.69	0.54	0.50
query37	0.10	0.07	0.08
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.04
Total cold run time: 103.81 s
Total hot run time: 29.52 s

Copy link
Contributor

@zzzxl1993 zzzxl1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Sep 6, 2025

PR approved by anyone and no changes requested.

@airborne12
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34621 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 90c84dd7293528a862b8b88a8b6706733fba5d27, data reload: false

------ Round 1 ----------------------------------
q1	17613	5123	5073	5073
q2	2013	313	209	209
q3	10262	1265	706	706
q4	10248	995	524	524
q5	7522	2391	2314	2314
q6	175	168	136	136
q7	920	745	612	612
q8	9364	1356	1128	1128
q9	6947	5071	5053	5053
q10	6940	2387	1999	1999
q11	493	315	281	281
q12	364	368	241	241
q13	17790	3689	3045	3045
q14	250	239	221	221
q15	570	497	494	494
q16	1007	1011	939	939
q17	609	857	361	361
q18	7517	7143	7118	7118
q19	1451	963	565	565
q20	338	336	240	240
q21	3757	3178	2383	2383
q22	1050	1026	979	979
Total cold run time: 107200 ms
Total hot run time: 34621 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5163	5057	5066	5057
q2	250	328	233	233
q3	2150	2681	2252	2252
q4	1358	1757	1317	1317
q5	4217	4571	4597	4571
q6	236	189	150	150
q7	2008	1942	1872	1872
q8	2671	2592	2609	2592
q9	7346	7370	7297	7297
q10	3146	3316	2866	2866
q11	577	513	496	496
q12	707	783	755	755
q13	3549	3877	3289	3289
q14	309	345	443	345
q15	524	505	474	474
q16	1085	1121	1097	1097
q17	1152	1532	1478	1478
q18	8211	8053	7608	7608
q19	776	774	1037	774
q20	1984	2008	1802	1802
q21	4631	4385	4233	4233
q22	1094	1029	993	993
Total cold run time: 53144 ms
Total hot run time: 51551 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188815 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 90c84dd7293528a862b8b88a8b6706733fba5d27, data reload: false

query1	1055	440	419	419
query2	6616	1694	1685	1685
query3	6756	226	230	226
query4	25914	23304	22937	22937
query5	4393	647	517	517
query6	340	265	236	236
query7	4648	516	308	308
query8	302	274	244	244
query9	8653	2879	2886	2879
query10	457	329	288	288
query11	15996	15014	14655	14655
query12	169	121	118	118
query13	1659	547	427	427
query14	10895	9097	9098	9097
query15	243	191	168	168
query16	7357	669	463	463
query17	1242	752	639	639
query18	2017	419	336	336
query19	205	200	171	171
query20	138	124	124	124
query21	224	133	113	113
query22	4195	4253	4044	4044
query23	34042	32910	33150	32910
query24	8145	2394	2390	2390
query25	602	538	476	476
query26	1241	277	177	177
query27	2732	510	358	358
query28	4409	2257	2221	2221
query29	846	625	521	521
query30	282	232	206	206
query31	931	818	782	782
query32	95	83	85	83
query33	598	411	399	399
query34	792	867	521	521
query35	817	835	764	764
query36	986	999	895	895
query37	139	116	96	96
query38	3524	3553	3470	3470
query39	1525	1464	1409	1409
query40	230	133	126	126
query41	65	62	65	62
query42	132	122	123	122
query43	512	504	470	470
query44	1358	851	887	851
query45	189	171	177	171
query46	852	1012	648	648
query47	1786	1851	1791	1791
query48	399	431	328	328
query49	743	511	411	411
query50	655	678	413	413
query51	3805	4094	3904	3904
query52	119	117	107	107
query53	250	278	195	195
query54	634	616	539	539
query55	97	93	92	92
query56	334	360	315	315
query57	1222	1194	1135	1135
query58	285	282	279	279
query59	2581	2639	2554	2554
query60	359	356	373	356
query61	169	162	161	161
query62	861	744	682	682
query63	231	198	195	195
query64	4484	1172	852	852
query65	4019	3972	3966	3966
query66	1182	439	373	373
query67	15479	15199	15233	15199
query68	9048	935	584	584
query69	486	384	299	299
query70	1383	1307	1274	1274
query71	563	352	316	316
query72	5825	4971	4970	4970
query73	744	633	360	360
query74	9077	9127	9090	9090
query75	4360	3232	2775	2775
query76	3663	1207	733	733
query77	802	409	327	327
query78	9517	9798	8878	8878
query79	4100	825	591	591
query80	678	587	528	528
query81	493	335	227	227
query82	648	167	138	138
query83	299	264	253	253
query84	301	107	91	91
query85	873	464	421	421
query86	355	313	318	313
query87	3792	3685	3630	3630
query88	2880	2236	2346	2236
query89	458	338	288	288
query90	1960	227	234	227
query91	167	163	135	135
query92	94	76	74	74
query93	2404	1000	655	655
query94	706	399	319	319
query95	415	340	335	335
query96	488	602	285	285
query97	2950	2967	2894	2894
query98	248	217	215	215
query99	1466	1412	1285	1285
Total cold run time: 279455 ms
Total hot run time: 188815 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 90c84dd7293528a862b8b88a8b6706733fba5d27, data reload: false

query1	0.05	0.04	0.05
query2	0.09	0.05	0.06
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.28	0.27	0.26
query6	1.19	0.68	0.63
query7	0.03	0.03	0.02
query8	0.05	0.05	0.05
query9	0.61	0.53	0.53
query10	0.58	0.57	0.58
query11	0.17	0.12	0.11
query12	0.16	0.12	0.12
query13	0.63	0.63	0.62
query14	1.02	1.05	1.03
query15	0.87	0.84	0.85
query16	0.41	0.42	0.41
query17	1.03	1.07	1.03
query18	0.22	0.20	0.22
query19	1.93	1.93	1.81
query20	0.01	0.01	0.01
query21	15.40	0.95	0.58
query22	0.76	1.18	0.86
query23	14.78	1.38	0.61
query24	6.58	1.62	0.38
query25	0.41	0.14	0.06
query26	0.59	0.17	0.15
query27	0.06	0.06	0.06
query28	9.59	0.99	0.43
query29	12.57	4.05	3.29
query30	0.28	0.13	0.14
query31	2.82	0.58	0.38
query32	3.24	0.56	0.49
query33	3.00	3.20	3.02
query34	16.02	5.47	4.92
query35	4.91	4.99	4.93
query36	0.69	0.52	0.50
query37	0.10	0.07	0.08
query38	0.06	0.04	0.05
query39	0.04	0.03	0.03
query40	0.20	0.15	0.15
query41	0.09	0.03	0.04
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 103.47 s
Total hot run time: 29.7 s

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 9, 2025
Copy link
Contributor

github-actions bot commented Sep 9, 2025

PR approved by at least one committer and no changes requested.

@airborne12
Copy link
Member Author

run check_coverage

@airborne12 airborne12 merged commit 2b605f8 into apache:master Sep 9, 2025
24 of 26 checks passed
@airborne12 airborne12 deleted the revert branch September 9, 2025 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants