.NET 9 Runtime Async Experiment

[agocke]

Update:

We've now completed the initial experiment into runtime-async. Overall, the experiment was very successful. For details, see https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md. We tested two possible implementations: a VM implementation and a JIT implementation. Of the two, we are more positive about the JIT implementation, both for performance and for maintenance.

Our primary conclusion is that runtime-async is at least as good as compiler-async in all the configurations that we measured. In addition, we believe that the new implementation can be fully compatible with the existing compiler-async, meaning that runtime-async can be a drop-in replacement.

We would like to graduate this experiment to a new runtime feature. However, this is a large feature which may take multiple releases to complete. In order to transparently replace the compiler-async implementation we will have to implement all the existing functionality in runtime-async.

For now we'll close out the experiment with the detailed results listed in the link above, and plan to publish more information on runtime-async planning as things become more concrete.

---

Intro

In .NET 8 we started an experiment in adding support for green threads to .NET. We learned a lot, but decided not to continue at the current time.

In .NET 9 we'd like to take what we learned and explore performance and usability improvements of the existing .NET async/Task threading model.

Background

From the C# and .NET libraries level, there are two supporting threading models: OS threads, and C# async/Task. Concurrent code can access each of these models using the System.Threading.Thread type or C# async/Task.Run, respectively. For most modern C# code we recommend using async and Task if you need blocking-free waiting or concurrency.

The Experiment

The status and code for the in-progress experiment can be found here: https://github.com/dotnet/runtimelab/tree/feature/async2-experiment

An ongoing design doc is present in: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md

While async and Task are the newest and most-preferred option at the C# and .NET libraries level, they are not a direct part of the .NET runtime threading model.

This experiment asks the question: what if they were? Rather than implement the async structure entirely in C# as a state machine rewrite, we are interested in exploring direct runtime integration with async methods.

The characteristics we're interested in for this experiment are:

• Throughput

  • Microbenchmarks -- how much does await cost?

  • Lots of nested awaits?

  • Frequent suspension vs. rare suspension

• Compatibility

  • Are the semantics similar/identical to C#?

  • Cost of switching

• Code size

  • IL size

  • Crossgen/Native AOT code size

As we explore more we might find more questions. At the moment, we're not planning to investigate things which require a lot of additional implementation work.

[hez2010]

FYI (while maybe unrelated as this issue is about the runtime, not the language), this (effect handlers) is how the newest way to do things like async, EH and etc. in a natural way (as for async, it doesn't require to split the APIs into async and sync parts).

https://effect-handlers.org/

You may see how they use effect handlers to implement async/await in C++ in the paper section.

[agocke]

I admit I'm only vaguely familiar with the effect typing literature. I've read part of Daniel Hillerström's dissertation, and looked through the Effekt language a bit.

I'm not sure there's anything directly interesting to the runtime there. In particular, the type system innovations I've seen in effect handlers seem to mostly be in the compiler front-end portion. Backend implementation appears to rely on a universal suspend-resume functionality implemented by the language runtime.

In this case we're interested in innovating directly in that suspend-resume primitive. And we're mostly interested in how it affects async in particular. We've found that the most pervasive effect that impacts performance is async, so we want to address it specifically.

For us to analyze a more general primitive I think we would want evidence that 1) that primitive is useful at the language level, namely that a language like C# would want first-class effects, and 2) the general-purpose primitive is equivalent in performance to the one we would implement for async. If (2) is not true then we would be leaving async performance on the table to handle user-defined effects, which currently seems like a bad trade-off.

[Clockwork-Muse]

... Midori did at least a throws/async effect primitive, mostly. They didn't generally implement an effect primitive (some notes near the bottom in Joe Duffy's blog about Midori's error model)

[jaredpar]

In Midori we did have async / throws modifiers to methods and they factored into the type system. It wasn't a full effect primitive though. For example you couldn't use the primitives in a generic fashion. The end effect (haha) is that for items like delegates you ended up with many varieties to support the combination of effects:

delegate void Action();
delegate throws void ActionThrows();
delegate async void ActionAsync();
delegate async throws void ActionThrowsAsync();

That is ... interesting but I don't think it will really impact this proposal very much.

[Jeevananthan-23]

Why not consider intergrating Rustlang to rewrite the coreCLR completely focus on Memory safety and Async throw deep. Somehow Microsoft is intergrating Rust os level. CC @davidfowl , @stephentoub

[davidfowl]

We like C# 😄

[lindexi]

@davidfowl Awesome. We're rooting for you!

[LeaFrock]

Do you think C# is worse that Rust in terms of "Memory safety and Async throw deep" 😕 ?