Use internal iteration in Vec::extend_desugared()

[ChayimFriedman2]

Because LLVM is unable to optimize well external iteration with some iterator kinds (e.g. chain()).

To do that I had to hoist the size_hint() call to the beginning of the loop (since I no longer have access to the iterator inside the loop), which might slightly pessimize certain iterators that are able to give more accurate size bounds during iteration (e.g. flatten()). However, the effect should not be big, and also, common iterators like these also suffer from the external iteration optimizibility problem (e.g. flatten()).

[rustbot]

r? @joboet

rustbot has assigned @joboet.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[compiler-errors]

Is there some sort of codegen test you could use to demonstrate that this has a beneficial effect?

[ChayimFriedman2]

I'm pretty sure a benchmark will show a difference, but I will check.

[rust-log-analyzer]

The job x86_64-gnu-llvm-18 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#19 exporting to docker image format
#19 sending tarball 19.8s done
#19 DONE 33.9s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-18]
[CI_JOB_NAME=x86_64-gnu-llvm-18]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Listening on address 127.0.0.1:4226
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-18', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'build.print-step-timings', '--enable-verbose-tests', '--set', 'build.metrics', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: build.build          := x86_64-unknown-linux-gnu
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-18/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
[RUSTC-TIMING] memchr test:false 0.900
error: variable does not need to be mutable
    --> library/alloc/src/vec/mod.rs:3538:59
     |
3538 |     fn extend_desugared<I: Iterator<Item = T>>(&mut self, mut iterator: I) {
     |                                                           ----^^^^^^^^
     |                                                           |
     |                                                           help: remove this `mut`
     |
     = note: `-D unused-mut` implied by `-D warnings`

[ChayimFriedman2]

So, @compiler-errors, unlike what I thought, this is not a 100% win (never trust your instincts in performance!). I benchmarked three scenarios: a flatten() call with a small array containing small slices, a call to flatten() with a large vector (100 elements) containing large vectors (0-100 elements, ascending), and small (15x10) vectors but with two chains and one flatten.

The results are: in the first two cases, the algorithms are almost equivalent, with a slight preference to the old algorithm for the first case and a slight preference to the new in the second case. In the third case, however, the new algorithm is almost 3x faster.

So my conclusion is: one such operation is not bad, but once you start to add more this version is significantly faster.

[joboet]

I think you could preserve the more precise length estimation by using try_for_each combined with Vec::push_within_capacity.