Skip to content

Conversation

yiguolei
Copy link
Contributor

@yiguolei yiguolei commented Sep 5, 2025

What problem does this PR solve?

If the input rows ==0 and the vertical_compaction_max_segment_size == 40g, the return value is 40g rows, it is larger than int32.max value, so that it will core.

But actually, the avg segment rows is always less or equal than the input rows num.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Sep 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yiguolei
Copy link
Contributor Author

yiguolei commented Sep 5, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34179 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f176f23be74e64c6049ff3c1d949c819b215e5f6, data reload: false

------ Round 1 ----------------------------------
q1	17608	5265	5022	5022
q2	1991	344	234	234
q3	10219	1547	741	741
q4	10244	1061	512	512
q5	7559	2340	2391	2340
q6	178	166	134	134
q7	920	753	648	648
q8	9347	1360	1105	1105
q9	7072	5152	5169	5152
q10	6892	2374	1963	1963
q11	524	319	282	282
q12	373	365	234	234
q13	17790	3633	3091	3091
q14	237	245	229	229
q15	572	493	516	493
q16	431	452	375	375
q17	621	875	356	356
q18	7636	7185	7150	7150
q19	1231	958	560	560
q20	343	348	232	232
q21	3780	2579	2318	2318
q22	1083	1069	1008	1008
Total cold run time: 106651 ms
Total hot run time: 34179 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5139	5079	5108	5079
q2	258	341	231	231
q3	2163	2745	2335	2335
q4	1335	1776	1328	1328
q5	4214	4315	4598	4315
q6	211	166	146	146
q7	2106	2005	1808	1808
q8	2667	2643	2598	2598
q9	7411	7417	7492	7417
q10	3148	3350	2808	2808
q11	640	509	522	509
q12	676	801	641	641
q13	3469	3897	3395	3395
q14	316	307	289	289
q15	530	480	480	480
q16	458	506	437	437
q17	1189	1638	1498	1498
q18	7774	7673	7620	7620
q19	817	838	897	838
q20	2081	2087	1963	1963
q21	5006	4270	4349	4270
q22	1112	1076	1028	1028
Total cold run time: 52720 ms
Total hot run time: 51033 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187276 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f176f23be74e64c6049ff3c1d949c819b215e5f6, data reload: false

query1	1029	469	405	405
query2	6562	1700	1734	1700
query3	6760	227	223	223
query4	26148	23408	23257	23257
query5	4391	655	516	516
query6	339	254	224	224
query7	4665	523	306	306
query8	305	282	261	261
query9	8647	2913	2911	2911
query10	518	374	305	305
query11	15471	15039	14736	14736
query12	184	124	131	124
query13	1697	577	455	455
query14	8919	5885	5817	5817
query15	214	204	175	175
query16	7805	679	495	495
query17	1224	760	628	628
query18	2046	499	346	346
query19	205	200	179	179
query20	130	128	125	125
query21	213	130	117	117
query22	4343	4284	4088	4088
query23	34040	32879	33151	32879
query24	8121	2357	2364	2357
query25	585	529	460	460
query26	1250	284	167	167
query27	2726	517	352	352
query28	4384	2266	2254	2254
query29	772	624	493	493
query30	292	227	201	201
query31	897	825	715	715
query32	95	85	80	80
query33	591	402	361	361
query34	808	871	527	527
query35	826	836	745	745
query36	962	1033	926	926
query37	126	111	93	93
query38	4079	4016	4030	4016
query39	1543	1446	1497	1446
query40	229	133	127	127
query41	67	71	65	65
query42	129	116	120	116
query43	554	516	475	475
query44	1358	860	863	860
query45	183	174	171	171
query46	850	1007	680	680
query47	1794	1856	1736	1736
query48	399	418	325	325
query49	756	529	424	424
query50	645	698	409	409
query51	4180	4270	4158	4158
query52	113	119	105	105
query53	251	269	205	205
query54	612	598	553	553
query55	97	90	92	90
query56	338	341	337	337
query57	1198	1216	1151	1151
query58	290	278	281	278
query59	2601	2668	2669	2668
query60	369	359	340	340
query61	170	161	163	161
query62	835	725	672	672
query63	239	193	191	191
query64	4434	1178	975	975
query65	4344	4269	4227	4227
query66	1125	461	374	374
query67	15477	15324	15294	15294
query68	8098	931	585	585
query69	521	397	292	292
query70	1252	1187	1085	1085
query71	564	365	324	324
query72	5833	5032	5234	5032
query73	745	687	368	368
query74	9141	9067	9090	9067
query75	3866	3109	2612	2612
query76	3697	1167	772	772
query77	805	404	332	332
query78	9607	9847	8856	8856
query79	2506	848	592	592
query80	686	580	524	524
query81	484	258	230	230
query82	434	142	111	111
query83	293	262	250	250
query84	301	122	94	94
query85	892	488	425	425
query86	345	322	316	316
query87	4265	4351	4146	4146
query88	3099	2265	2252	2252
query89	413	334	303	303
query90	1915	231	224	224
query91	167	188	131	131
query92	97	80	77	77
query93	1451	993	637	637
query94	697	431	331	331
query95	423	340	338	338
query96	491	584	281	281
query97	2652	2699	2555	2555
query98	252	221	219	219
query99	1484	1471	1283	1283
Total cold run time: 275025 ms
Total hot run time: 187276 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.21 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f176f23be74e64c6049ff3c1d949c819b215e5f6, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.25	0.09	0.09
query4	1.61	0.11	0.12
query5	0.44	0.41	0.40
query6	1.18	0.65	0.65
query7	0.04	0.03	0.03
query8	0.06	0.04	0.04
query9	0.60	0.53	0.52
query10	0.59	0.58	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.13
query13	0.63	0.63	0.61
query14	0.80	0.83	0.84
query15	0.88	0.86	0.86
query16	0.39	0.40	0.39
query17	1.02	1.06	1.04
query18	0.22	0.21	0.21
query19	1.88	1.83	1.81
query20	0.02	0.01	0.02
query21	15.43	0.96	0.58
query22	0.76	1.26	0.95
query23	14.78	1.39	0.62
query24	6.37	1.24	0.84
query25	0.48	0.18	0.07
query26	0.58	0.16	0.14
query27	0.06	0.05	0.06
query28	9.78	0.96	0.44
query29	12.59	3.95	3.29
query30	0.28	0.13	0.12
query31	2.83	0.62	0.39
query32	3.26	0.56	0.48
query33	3.08	3.10	3.19
query34	16.08	5.47	4.88
query35	4.90	4.86	4.86
query36	0.69	0.52	0.54
query37	0.10	0.07	0.08
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 103.59 s
Total hot run time: 30.21 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 50.00% (3/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 51.88% (17231/33212)
Line Coverage 37.28% (157220/421758)
Region Coverage 31.90% (119948/376013)
Branch Coverage 33.29% (52671/158232)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 70.62% (23031/32612)
Line Coverage 56.96% (240060/421482)
Region Coverage 52.32% (199548/381400)
Branch Coverage 54.00% (85925/159132)

Copy link
Contributor

github-actions bot commented Sep 5, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2025
Copy link
Contributor

github-actions bot commented Sep 5, 2025

PR approved by anyone and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 371ab36 into apache:master Sep 6, 2025
27 of 30 checks passed
wenzhenghu pushed a commit to wenzhenghu/doris that referenced this pull request Sep 8, 2025
…input rows (apache#55713)

### What problem does this PR solve?

If the input rows ==0 and the vertical_compaction_max_segment_size ==
40g, the return value is 40g rows, it is larger than int32.max value, so
that it will core.

But actually, the avg segment rows is always less or equal than the
input rows num.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants