Use a thread pool when building runtimes #6133

chandlerc · 2025-09-25T14:12:08Z

This parallelizes the compilations and dramatically reduces the time to build runtimes.

As part of this, teach the driver infrastructure to have an option to control the use of threads and to build the relevant thread pool and thread it into the various APIs.

However, it requires our ClangRunner to become thread-safe and to invoke Clang in a way that is thread-safe. This is somewhat challenging as the code in clang_main is distinctly not thread-safe.

To address this, the relevant logic of clang_main, especially the CC1 execution, is extracted into our runner and cleaned up to be much more appropriate in a multithreaded context. Much of this code should eventually be factored back into Clang, but that will be a follow-up patch to upstream.

Last but not least, this rearranges the ClangRunner API to make a bit more sense out of the different options for building runtimes, and have a clean model for which things need to be passed in at which points.

This parallelizes the compilations and dramatically reduces the time to build runtimes. As part of this, teach the driver infrastructure to have an option to control the use of threads and to build the relevant thread pool and thread it into the various APIs. However, it requires our `ClangRunner` to become thread-safe and to invoke Clang in a way that is thread-safe. This is somewhat challenging as the code in `clang_main` is distinctly _not_ thread-safe. To address this, the relevant logic of `clang_main`, especially the CC1 execution, is extracted into our runner and cleaned up to be much more appropriate in a multithreaded context. Much of this code should eventually be factored back into Clang, but that will be a follow-up patch to upstream.

toolchain/driver/clang_runner.cpp

toolchain/driver/clang_runner_test.cpp

danakj · 2025-09-25T15:07:48Z

toolchain/driver/driver.cpp

+  llvm::SingleThreadExecutor single_thread({.ThreadsRequested = 1});
+  std::optional<llvm::DefaultThreadPool> threads;
+  driver_env_.thread_pool = &single_thread;
+  if (options.threads) {
+    threads.emplace(llvm::optimal_concurrency());
+    driver_env_.thread_pool = &*threads;
+  }


We're are creating this SingleThreadExecutor at multiple levels it seems, both here and inside the ClangRunner. Could we consolidate to a single place - maybe making ClangRunner expect to receive a non-null executor always?

We could... I was just a bit torn doing so as it adds quite a bit of complexity to building and using the runner which is only needed if you're actually building runtimes.

Another alternative would be to have ClangRunner either accept a pre-built path or a thread pool to use for on-demand building, and remove the boolean option.

Do you forsee wanting to use other thread pools than SingleThreadExecutor and DefaultThreadPool? Maybe we could tell it if we want threads or not and have ClangRunner make what it needs?

enum class BuildRuntimesOnDemand { UsePrebuiltOnly, BuildOnSingleThreaded, BuildOnWorkerThreads, }

Taking either a path XOR a thread pool also sounds good, whichever you pref

Do you forsee wanting to use other thread pools than SingleThreadExecutor and DefaultThreadPool?

Some possibility. One thing I wonder is if we'll want a (much) larger thread pool to fully absorb the latency, but would like to avoid it given the overhead.

Maybe we could tell it if we want threads or not and have ClangRunner make what it needs?

When using threads, my expectation is that it'll be very desirable to use the existing thread pool to avoid paying the cost of spinning one up and forking all the threads.

It also allows more global management of the load.

This is why I somewhat like the driver either accepting or building a thread pool, and then making it available for any commands or subcommands to use.

Taking either a path XOR a thread pool also sounds good, whichever you pref

I think what I'm liking is to take one of:

A thread pool, enabling on-demand building in that pool.

A pre-built path that will be used.

Nothing, disabling on-demand building, and its up to the caller to use the runner in a way compatible with that.

Actually, I thought more about this and I think I had the fundamental wrong structure of this API.

I've restructured everything so that we have a simple constructor that only accepts the necessary components. Then there are three variations on Run -- using on-demand runtimes with a cache and thread pool, using pre-built runtimes, and using no runtimes. I've also updated comments and callers accourdingly. I think this ends up more clear, but PTAL and let me know.

toolchain/driver/clang_runner.cpp

Co-authored-by: Dana Jansens <[email protected]>

chandlerc

Thanks, PTAL!

toolchain/driver/clang_runner_test.cpp

chandlerc · 2025-09-25T15:53:32Z

toolchain/driver/driver.cpp

+  llvm::SingleThreadExecutor single_thread({.ThreadsRequested = 1});
+  std::optional<llvm::DefaultThreadPool> threads;
+  driver_env_.thread_pool = &single_thread;
+  if (options.threads) {
+    threads.emplace(llvm::optimal_concurrency());
+    driver_env_.thread_pool = &*threads;
+  }


We could... I was just a bit torn doing so as it adds quite a bit of complexity to building and using the runner which is only needed if you're actually building runtimes.

Another alternative would be to have ClangRunner either accept a pre-built path or a thread pool to use for on-demand building, and remove the boolean option.

toolchain/driver/clang_runner.cpp

danakj · 2025-09-26T13:31:50Z

toolchain/driver/clang_runner.cpp

+  llvm::SmallVector<llvm::NewArchiveMember, 0> unwrapped_objs;
+  unwrapped_objs.reserve(objs.size());
+  for (auto& obj : objs) {
+    unwrapped_objs.push_back(*std::move(obj));
+  }


If you enjoy some code golf...

llvm::SmallVector<llvm::NewArchiveMember, 0> unwrapped_objs( llvm::map_range(objs, [](auto& obj) { return *std::move(obj); }));

Yeah, I somewhat prefer the simplicity of the loop here... It's a close call though, happy to switch if it helps a lot.

toolchain/driver/clang_runner.cpp

toolchain/driver/clang_runner_test.cpp

danakj · 2025-09-26T13:36:44Z

toolchain/driver/driver.cpp

+  llvm::SingleThreadExecutor single_thread({.ThreadsRequested = 1});
+  std::optional<llvm::DefaultThreadPool> threads;
+  driver_env_.thread_pool = &single_thread;
+  if (options.threads) {
+    threads.emplace(llvm::optimal_concurrency());
+    driver_env_.thread_pool = &*threads;
+  }


Do you forsee wanting to use other thread pools than SingleThreadExecutor and DefaultThreadPool? Maybe we could tell it if we want threads or not and have ClangRunner make what it needs?

enum class BuildRuntimesOnDemand { UsePrebuiltOnly, BuildOnSingleThreaded, BuildOnWorkerThreads, }

Taking either a path XOR a thread pool also sounds good, whichever you pref

Co-authored-by: Dana Jansens <[email protected]>

chandlerc

PTAL, think I found a better API structure.

toolchain/driver/clang_runner.cpp

chandlerc · 2025-09-26T13:52:29Z

toolchain/driver/clang_runner.cpp

+  llvm::SmallVector<llvm::NewArchiveMember, 0> unwrapped_objs;
+  unwrapped_objs.reserve(objs.size());
+  for (auto& obj : objs) {
+    unwrapped_objs.push_back(*std::move(obj));
+  }


Yeah, I somewhat prefer the simplicity of the loop here... It's a close call though, happy to switch if it helps a lot.

chandlerc · 2025-09-26T13:58:17Z

toolchain/driver/driver.cpp

+  llvm::SingleThreadExecutor single_thread({.ThreadsRequested = 1});
+  std::optional<llvm::DefaultThreadPool> threads;
+  driver_env_.thread_pool = &single_thread;
+  if (options.threads) {
+    threads.emplace(llvm::optimal_concurrency());
+    driver_env_.thread_pool = &*threads;
+  }


Do you forsee wanting to use other thread pools than SingleThreadExecutor and DefaultThreadPool?

Some possibility. One thing I wonder is if we'll want a (much) larger thread pool to fully absorb the latency, but would like to avoid it given the overhead.

Maybe we could tell it if we want threads or not and have ClangRunner make what it needs?

When using threads, my expectation is that it'll be very desirable to use the existing thread pool to avoid paying the cost of spinning one up and forking all the threads.

It also allows more global management of the load.

This is why I somewhat like the driver either accepting or building a thread pool, and then making it available for any commands or subcommands to use.

Taking either a path XOR a thread pool also sounds good, whichever you pref

I think what I'm liking is to take one of:

A thread pool, enabling on-demand building in that pool.

A pre-built path that will be used.

Nothing, disabling on-demand building, and its up to the caller to use the runner in a way compatible with that.

chandlerc · 2025-09-27T02:06:36Z

toolchain/driver/driver.cpp

+  llvm::SingleThreadExecutor single_thread({.ThreadsRequested = 1});
+  std::optional<llvm::DefaultThreadPool> threads;
+  driver_env_.thread_pool = &single_thread;
+  if (options.threads) {
+    threads.emplace(llvm::optimal_concurrency());
+    driver_env_.thread_pool = &*threads;
+  }


Actually, I thought more about this and I think I had the fundamental wrong structure of this API.

I've restructured everything so that we have a simple constructor that only accepts the necessary components. Then there are three variations on Run -- using on-demand runtimes with a cache and thread pool, using pre-built runtimes, and using no runtimes. I've also updated comments and callers accourdingly. I think this ends up more clear, but PTAL and let me know.

danakj

Yes the new API looks like a really nice improvement, thanks. LGTM

toolchain/driver/clang_runner.cpp

Co-authored-by: Dana Jansens <[email protected]>

toolchain/driver/clang_runner.cpp

github-actions bot requested a review from danakj September 25, 2025 14:12

github-actions bot added the toolchain label Sep 25, 2025

danakj reviewed Sep 25, 2025

View reviewed changes

chandlerc and others added 4 commits September 25, 2025 17:47

disable more free

a732ec4

Apply suggestions from code review

40f5e61

Co-authored-by: Dana Jansens <[email protected]>

review feedback and fixes

df546ea

format

cc7cb77

chandlerc commented Sep 25, 2025

View reviewed changes

chandlerc requested a review from danakj September 25, 2025 23:34

danakj reviewed Sep 26, 2025

View reviewed changes

chandlerc and others added 3 commits September 26, 2025 06:52

Apply suggestions from code review

94370ce

Co-authored-by: Dana Jansens <[email protected]>

tweak

1f90e49

adjust API

13289a2

chandlerc commented Sep 27, 2025

View reviewed changes

danakj approved these changes Sep 29, 2025

View reviewed changes

toolchain/driver/clang_runner.cpp Outdated Show resolved Hide resolved

Update toolchain/driver/clang_runner.cpp

5fbb837

Co-authored-by: Dana Jansens <[email protected]>

CarbonInfraBot reviewed Sep 30, 2025

View reviewed changes

toolchain/driver/clang_runner.cpp Outdated Show resolved Hide resolved

fixes

1833476

chandlerc enabled auto-merge September 30, 2025 07:14

chandlerc added this pull request to the merge queue Sep 30, 2025

github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Sep 30, 2025

danakj added this pull request to the merge queue Sep 30, 2025

Merged via the queue into carbon-language:trunk with commit 35fb000 Sep 30, 2025
8 checks passed

Use a thread pool when building runtimes #6133

Use a thread pool when building runtimes #6133

Uh oh!

Conversation

chandlerc commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chandlerc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chandlerc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danakj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chandlerc commented Sep 25, 2025 •

edited

Loading