[perf] split: reuse the per-line buffer in round-robin mode - #14790
Merged
sylvestre merged 1 commit intoSep 22, 2026
Merged
Conversation
sylvestre
reviewed
Sep 22, 2026
| let mut closed_writers = 0; | ||
|
|
||
| let mut i = 0; | ||
| let mut line = Vec::new(); |
Contributor
There was a problem hiding this comment.
could you please add a bench for the r/N mode in benches/split_bench.rs? otherwise codspeed won't track this gain
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Small improvement today.
Moving the allocation out of the loop, and reuse the vec instead of allocating and freeing it every line. Because it's allocated on every line, it will stretch (reallocate) if the line length exceeds the allocated buffer.
Clanker generated:
Vec Mechanics recap:
So for read_until reusing the buffer: it grows through the doubling sequence only while a single line is being read and only until it fits that line; thereafter clear() keeps whatever
capacity that line reached, and shorter subsequent lines reuse it with zero realloc. The 64-byte figure in my earlier answer was just an illustration of "a capacity value in the growth
series," not a fixed allocation size.
Verification (A/B on this branch,
mainvssplit-round-robin-vec-outside-loop)Release fat-LTO
coreutilsbinaries,split --number=r/4on a 500k-line file:maincargo test -p uu_splitpasses.