Initial support for auto traits with default bounds #120706

Bryanskiy · 2024-02-06T16:10:29Z

This PR is part of "MCP: Low level components for async drop"
Summary: #120706 (comment)

Intro

Sometimes we want to use type system to express specific behavior and provide safety guarantees. This behavior can be specified by various "marker" traits. For example, we use Send and Sync to keep track of which types are thread safe. As the language develops, there are more problems that could be solved by adding new marker traits:

to forbid types with an async destructor to be dropped in a synchronous context a trait like SyncDrop could be used Async destructors, async genericity and completion futures.
to support scoped tasks or in a more general sense to provide a destruction guarantee there is a desire among some users to see a Leak (or Forget) trait.
Withoutboats in his post reflected on the use of Move trait instead of a Pin.

All the traits proposed above are supposed to be auto traits implemented for most types, and usually implemented automatically by compiler.

For backward compatibility these traits have to be added implicitly to all bound lists in old code (see below). Adding new default bounds involves many difficulties: many standard library interfaces may need to opt out of those default bounds, and therefore be infected with confusing ?Trait syntax, migration to a new edition may contain backward compatibility holes, supporting new traits in the compiler can be quite difficult and so forth. Anyway, it's hard to evaluate the complexity until we try the system on a practice.

In this PR we introduce new optional lang items for traits that are added to all bound lists by default, similarly to existing Sized. The examples of such traits could be Leak, Move, SyncDrop or something else, it doesn't matter much right now (further I will call them DefaultAutoTrait's). We want to land this change into rustc under an option, so it becomes available in bootstrap compiler. Then we'll be able to do standard library experiments with the aforementioned traits without adding hundreds of #[cfg(not(bootstrap))]s. Based on the experiments, we can come up with some scheme for the next edition, in which such bounds are added in a more targeted way, and not just everywhere.

Most of the implementation is basically a refactoring that replaces hardcoded uses of Sized with iterating over a list of traits including both Sized and the new traits when -Zexperimental-default-bounds is enabled (or just Sized as before, if the option is not enabled).

Default bounds for old editions

All existing types, including generic parameters, are considered Leak/Move/SyncDrop and can be forgotten, moved or destroyed in generic contexts without specifying any bounds. New types that cannot be, for example, forgotten and do not implement Leak can be added at some point, and they should not be usable in such generic contexts in existing code.

To both maintain this property and keep backward compatibility with existing code, the new traits should be added as default bounds everywhere in previous editions. Besides the implicit Sized bound contexts that includes supertrait lists and trait lists in trait objects (dyn Trait1 + ... + TraitN). Compiler should also generate implicit DefaultAutoTrait implementations for foreign types (extern { type Foo; }) because they are also currently usable in generic contexts without any bounds.

Supertraits

Adding the new traits as supertraits to all existing traits is potentially necessary, because, for example, using a Self param in a trait's associated item may be a breaking change otherwise:

trait Foo: Sized {
    fn new() -> Option<Self>; // ERROR: `Option` requires `DefaultAutoTrait`, but `Self` is not `DefaultAutoTrait`
}

// desugared `Option`
enum Option<T: DefaultAutoTrait + Sized> {
    Some(T),
    None,
}

However, default supertraits can significantly affect compiler performance. For example, if we know that T: Trait, the compiler would deduce that T: DefaultAutoTrait. It also implies proving F: DefaultAutoTrait for each field F of type T until an explicit impl is be provided.

If the standard library is not modified, then even traits like Copy or Send would get these supertraits.

In this PR for optimization purposes instead of adding default supertraits, bounds are added to the associated items:

// Default bounds are generated in the following way:
trait Trait {
   fn foo(&self) where Self: DefaultAutoTrait {}
}

// instead of this:
trait Trait: DefaultAutoTrait {
   fn foo(&self) {}
}

It is not always possible to do this optimization because of backward compatibility:

pub trait Trait<Rhs = Self> {}
pub trait Trait1 : Trait {} // ERROR: `Rhs` requires `DefaultAutoTrait`, but `Self` is not `DefaultAutoTrait`

or

trait Trait {
   type Type where Self: Sized;
}
trait Trait2<T> : Trait<Type = T> {} // ERROR: `???` requires `DefaultAutoTrait`, but `Self` is not `DefaultAutoTrait`

Therefore, DefaultAutoTrait's are still being added to supertraits if the Self params or type bindings were found in the trait header.

Trait objects

Trait objects requires explicit + Trait bound to implement corresponding trait which is not backward compatible:

fn use_trait_object(x: Box<dyn Trait>) {
   foo(x) // ERROR: `foo` requires `DefaultAutoTrait`, but `dyn Trait` is not `DefaultAutoTrait`
}

// implicit T: DefaultAutoTrait here
fn foo<T>(_: T) {}

So, for a trait object dyn Trait we should add an implicit bound dyn Trait + DefaultAutoTrait to make it usable, and allow relaxing it with a question mark syntax dyn Trait + ?DefaultAutoTrait when it's not necessary.

Foreign types

If compiler doesn't generate auto trait implementations for a foreign type, then it's a breaking change if the default bounds are added everywhere else:

// implicit T: DefaultAutoTrait here
fn foo<T: ?Sized>(_: &T) {}

extern "C" {
    type ExternTy;
}

fn forward_extern_ty(x: &ExternTy) {
    foo(x); // ERROR: `foo` requires `DefaultAutoTrait`, but `ExternTy` is not `DefaultAutoTrait`
}

We'll have to enable implicit DefaultAutoTrait implementations for foreign types at least for previous editions:

// implicit T: DefaultAutoTrait here
fn foo<T: ?Sized>(_: &T) {}

extern "C" {
    type ExternTy;
}

impl DefaultAutoTrait for ExternTy {} // implicit impl

fn forward_extern_ty(x: &ExternTy) {
    foo(x); // OK
}

Unresolved questions

New default bounds affect all existing Rust code complicating an already complex type system.

Proving an auto trait predicate requires recursively traversing the type and proving the predicate for it's fields. This leads to a significant performance regression. Measurements for the stage 2 compiler build show up to 3x regression.
- We hope that fast path optimizations for well known traits could mitigate such regressions at least partially.
New default bounds trigger some compiler bugs in both old and new trait solver.
With new default bounds we encounter some trait solver cycle errors that break existing code.
- We hope that these cases are bugs that can be addressed in the new trait solver.

Also migration to a new edition could be quite ugly and enormous, but that's actually what we want to solve. For other issues there's a chance that they could be solved by a new solver.

Bryanskiy · 2024-02-06T16:10:42Z

@rustbot author

petrochenkov · 2024-02-08T12:23:07Z

So, what are the goals here:

We want to have a possibility to add new auto traits that are added to all bound lists by default on the current edition. The examples of such traits could be Leak, Move, SyncDrop or something else, it doesn't matter much right now. The desired behavior is similar to the current Sized trait. Such behavior is required for introducing !Leak or !SyncDrop types in a backward compatible way. (Both Leak and SyncDrop are likely necessary for properly supporting libraries for scoped async tasks and structured concurrency.)
It's not clear whether it can be done backward compatibly and without significant perf regressions, but that's exactly what we want to find out. Right now we encounter some cycle errors and exponential blow ups in the trait solver, but there's a chance that they are fixable with the new solver.
Then we want to land the change into rustc under an option, so it becomes available in bootstrap compiler. Then we'll be able to do standard library experiments with the aforementioned traits without adding hundreds of #[cfg(not(bootstrap))]s.
Based on the experiments, we can come up with some scheme for the next edition, in which such bounds are added more conservatively.
Relevant blog posts - https://without.boats/blog/changing-the-rules-of-rust/, https://without.boats/blog/follow-up-to-changing-the-rules-of-rust/ and https://without.boats/blog/generic-trait-methods-and-new-auto-traits/, https://without.boats/blog/the-scoped-task-trilemma/
Larger compiler team MCP including this feature - MCP: Low level components for async drop compiler-team#727, it gives some more context

petrochenkov · 2024-02-08T12:26:51Z

The issue right now is that there are regressions, some previously passing code now fails due to cycles in trait solver or something similar, @Bryanskiy has been trying to investigate it, but without success.

@lcnr, this is the work I've been talking about today.
(Maybe it makes sense to ping some other types team members as well?)

compiler/rustc_hir/src/lang_items.rs

Bryanskiy · 2024-02-08T12:39:34Z

I want to reproduce regressions in CI

@rustbot author

lcnr · 2024-02-08T13:25:39Z

it would be good to get an MVCE for the new cycles, otherwise debugging this without fully going through the PR is hard

Support ?Trait bounds in supertraits and dyn Trait under a feature gate This patch allows `maybe` polarity bounds under a feature gate. The only language change here is that corresponding hard errors are replaced by feature gates. Example: ```rust #![feature(allow_maybe_polarity)] ... trait Trait1 : ?Trait { ... } // ok fn foo(_: Box<(dyn Trait2 + ?Trait)>) {} // ok fn bar<T: ?Sized + ?Trait>(_: &T) {} // ok ``` Maybe bounds still don't do anything (except for `Sized` trait), however this patch will allow us to [experiment with default auto traits](rust-lang#120706 (comment)). This is a part of the [MCP: Low level components for async drop](rust-lang/compiler-team#727)

Support ?Trait bounds in supertraits and dyn Trait under a feature gate This patch allows `maybe` polarity bounds under a feature gate. The only language change here is that corresponding hard errors are replaced by feature gates. Example: ```rust #![feature(allow_maybe_polarity)] ... trait Trait1 : ?Trait { ... } // ok fn foo(_: Box<(dyn Trait2 + ?Trait)>) {} // ok fn bar<T: ?Sized + ?Trait>(_: &T) {} // ok ``` Maybe bounds still don't do anything (except for `Sized` trait), however this patch will allow us to [experiment with default auto traits](rust-lang/rust#120706 (comment)). This is a part of the [MCP: Low level components for async drop](rust-lang/compiler-team#727)

Bryanskiy · 2024-08-20T13:29:07Z

@rustbot ready

bors · 2024-08-27T23:46:13Z

☔ The latest upstream changes (presumably #129665) made this pull request unmergeable. Please resolve the merge conflicts.

Ddystopia · 2025-02-07T11:00:33Z

I didn't know about this effort and wrote a Pre-RFC for exact same use case... I guess it is not needed, considering this? I'll attach it there
0000-local-default-generic-bounds.md

(gist)

zetanumbers · 2025-02-07T13:05:06Z

I didn't know about this effort and wrote a Pre-RFC for exact same use case... I guess it is not needed, considering this? I'll attach it there 0000-local-default-generic-bounds.md

(gist)

I appreciate your effort of formalizing default generic bounds for new auto-traits. This is something I've planned to do, but I've struggled to solve one problem that was stated to me by decision makers, as to add a new auto-trait would require reducing ergonomics of existing libraries, that obviously don't implement these auto-traits, thus making experience of using those libraries or auto-traits unergonomic.

I am working on something, that could potentially partially resolve this issue, but it also may be too complex of a change for the Rust language project anyway.

zetanumbers · 2025-02-07T13:19:19Z

This is something I've planned to do, but I've struggled to solve one problem that was stated to me by decision makers, as to add a new auto-trait would require reducing ergonomics of existing libraries, that obviously don't implement these auto-traits, thus making experience of using those libraries or auto-traits unergonomic.

I have tried to make a more specific argument, that adding the Forget auto-trait would bring more benefit, than the cost of old libraries to not implement its support. You can try argue against the ergonomics point in your RFC by diving into specifics of proposed auto-traits. For that you can read my post about unforgettable types, for which this PR was made for:

https://zetanumbers.github.io/book/myosotis.html

Anyways, thank you. This suggestion wasn't written down before to this level of detail you have provided.

LFS6502 · 2025-03-01T10:00:22Z

@Bryanskiy any update on this PR? Thanks.

Bryanskiy · 2025-03-01T15:24:20Z

@LFS6502 Hi, I'll come back to this PR next week.

Bryanskiy · 2025-03-08T10:56:26Z

@rustbot ready

lcnr

some nits for this PR, otherwise it seems good given that it's all still gated behind a feature flag.

As you mention in your doc: adding default auto traits as super trait bounds causes a huge performance costs (mostly) by increasing the ParamEnv size, it'll also cause a huge performance cost by just having to prove the auto trait bounds themselves. What's worse is that it will also just cause code to hit the recursion limit, causing many crates with deeply nested types to require changes.

Adding a fast path for auto traits which does not result in undesirable breakage or is difficult to reason about and may be unsound seems challenging.

I am personally very doubtful we will ever be able to stabilize additional default traits unless they stop at indirection like Sized. I am fine with experimentation here, but do not want us to land any changes for it which are difficult to revert or greatly increase the complexity of code paths also used on stable/by other features. I would also like you to be careful when talking about this feature to make sure people are aware that actually stabilizing such traits will be challenging.

lcnr · 2025-03-10T15:40:17Z

compiler/rustc_hir_analysis/src/collect/predicates_of.rs

            }
            _ => {}
        }
    };

+    if let Node::TraitItem(item) = node {


do not put this in gather_explicit_predicates_of. This is for predicates written by the user. Push this one level out to the caller

It's not clear to me. From what I see Sized bounds are added in gather_explicit_predicates_of, but they aren't explicit in the ordinary sense. Do they considered explicit because of the user written T: ?Sized bounds? For default_auto_traits we have the same situation, except that T: ?Trait bounds can be used in more contexts, and here is one such context.

Perhaps you were confused by the implicit word in new methods from bounds.rs. Renamed them.

lcnr · 2025-03-10T15:41:42Z

compiler/rustc_hir_analysis/src/hir_ty_lowering/bounds.rs

+        // FIXME(experimental_default_bounds): Default bounds on `Pointee::Metadata`
+        // causes a normalization fail.
+        if Some(trait_def_id.to_def_id()) == self.tcx().lang_items().pointee_trait() {
+            return true;
+        }


It feels questionable to land this experimentation as long as this bug exists. It needs an explanation for why it won't affect user written impls as these bounds are otherwise a likely unacceptable breaking change

A double check showed that the bug had been fixed.

lcnr · 2025-03-10T15:42:24Z

compiler/rustc_hir_analysis/src/hir_ty_lowering/bounds.rs

+        );
+    }
+
+    pub(crate) fn requires_implicit_supertraits(


this very much needs an explanation of what it's doing and why

lcnr · 2025-03-10T15:48:05Z

compiler/rustc_hir_analysis/src/hir_ty_lowering/bounds.rs

+    /// For optimization purposes instead of adding default supertraits, bounds
+    /// are added to the associative items:


Suggested change

/// For optimization purposes instead of adding default supertraits, bounds

/// are added to the associative items:

/// For optimization purposes instead of adding default supertraits, bounds

/// are added to the associated items:

lcnr · 2025-03-10T15:51:02Z

compiler/rustc_hir_analysis/src/hir_ty_lowering/bounds.rs

+        }
+    }
+
+    fn check_for_implicit_trait(


this bool return type is confusing without reading the source, please change the fn name to something like "do_not_provide_implicit_trait_bound" (in this case, invert the condition to avoid an unnecessary negation) or change the return type to a newtyped enum

lcnr · 2025-03-10T15:53:24Z

compiler/rustc_lint/src/multiple_supertrait_upcastable.rs

@@ -46,7 +46,8 @@ impl<'tcx> LateLintPass<'tcx> for MultipleSupertraitUpcastable {
                .tcx
                .explicit_super_predicates_of(def_id)
                .iter_identity_copied()
-                .filter_map(|(pred, _)| pred.as_trait_clause());
+                .filter_map(|(pred, _)| pred.as_trait_clause())
+                .filter(|pred| !cx.tcx.is_default_trait(pred.def_id()));


The hack for old editions. The dyn-compatible trait with multiple DefaultAutoTrait super bounds will trigger the lint. But in old editions we can't relax bounds with ?DefaultAutoTrait syntax, so it can be annoying.

But yeah, it's better to remove it.

lcnr · 2025-03-10T15:57:22Z

compiler/rustc_next_trait_solver/src/solve/assembly/structural_traits.rs

@@ -31,6 +31,7 @@ where
        | ty::Float(_)
        | ty::FnDef(..)
        | ty::FnPtr(..)
+        | ty::Foreign(..)


please keep this in a separate branch and add a comment/debug_assert that it only affects the default auto traits

lcnr · 2025-03-10T15:58:32Z

compiler/rustc_next_trait_solver/src/solve/trait_goals.rs

@@ -1096,6 +1115,10 @@ where
                Some(self.forced_ambiguity(MaybeCause::Ambiguity))
            }

+            // Backward compatibility for default auto traits.
+            // Test: ui/traits/default_auto_traits/extern-types.rs
+            ty::Foreign(..) if self.cx().is_default_trait(goal.predicate.def_id()) => check_impls(),


this feels undesirable. Foreign types should not implement Leak/Move, should they?

given that we don't enable the feature on stable, I feel like this hack should be removed

Foreign types from old editions should implement DefaultAutoTrait's to keep backward compatibility(I have example in explainer). For new editions, they shouldn't, but an edition-based migration mechanism has not yet been designed/implemented.

lcnr · 2025-03-10T15:59:07Z

compiler/rustc_trait_selection/src/traits/dyn_compatibility.rs

@@ -598,8 +599,26 @@ fn receiver_is_dispatchable<'tcx>(
            ty::TraitRef::new_from_args(tcx, trait_def_id, args).upcast(tcx)
        };

-        let caller_bounds =
-            param_env.caller_bounds().iter().chain([unsize_predicate, trait_predicate]);
+        // U: `experimental_default_bounds`


One more hack for std build. Implicit T: DefaultAutoTrait in DispatchFromDyn<T> produces U: DefaultAutoTrait goal. Have been removed.

lcnr · 2025-03-10T16:17:20Z

compiler/rustc_session/src/options.rs

@@ -2180,6 +2180,8 @@ options! {
        "Use WebAssembly error handling for wasm32-unknown-emscripten"),
    enforce_type_length_limit: bool = (false, parse_bool, [TRACKED],
        "enforce the type length limit when monomorphizing instances in codegen"),
+    experimental_default_bounds: bool = (false, parse_bool, [TRACKED],


please open a tracking issue for this and list the current known issues

bors · 2025-03-12T11:14:28Z

☔ The latest upstream changes (presumably #138388) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot assigned petrochenkov Feb 6, 2024

rustbot added S-waiting-on-review T-compiler T-libs WG-trait-system-refactor labels Feb 6, 2024

rustbot added S-waiting-on-author and removed S-waiting-on-review labels Feb 6, 2024

This comment has been minimized.

Sign in to view

Bryanskiy force-pushed the leak branch from ad1a725 to f5181dd Compare February 6, 2024 16:21

This comment has been minimized.

Sign in to view

Bryanskiy force-pushed the leak branch from f5181dd to df3a6ab Compare February 6, 2024 16:47

This comment has been minimized.

Sign in to view

Bryanskiy force-pushed the leak branch from df3a6ab to c9d15c4 Compare February 7, 2024 18:21

This comment has been minimized.

Sign in to view

Bryanskiy force-pushed the leak branch from 96b5ea1 to 3f57f13 Compare February 8, 2024 12:17

rustbot added A-testsuite T-bootstrap labels Feb 8, 2024

This comment has been minimized.

Sign in to view

petrochenkov reviewed Feb 8, 2024

View reviewed changes

compiler/rustc_hir/src/lang_items.rs Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-review and removed S-waiting-on-author labels Feb 8, 2024

petrochenkov assigned lcnr and unassigned petrochenkov Feb 8, 2024

rustbot added S-waiting-on-author and removed S-waiting-on-review labels Feb 8, 2024

Dylan-DPC added S-waiting-on-author and removed S-waiting-on-review labels Aug 7, 2024

Bryanskiy force-pushed the leak branch from a988faa to 5bb36db Compare August 20, 2024 12:06

rustbot added S-waiting-on-review and removed S-waiting-on-author labels Aug 20, 2024

apiraino removed the S-waiting-on-team label Aug 29, 2024

lcnr added S-waiting-on-author and removed S-waiting-on-review labels Jan 21, 2025

Bryanskiy force-pushed the leak branch from 5bb36db to 6c33a56 Compare March 7, 2025 17:38

rustbot added S-waiting-on-review and removed S-waiting-on-author labels Mar 8, 2025

lcnr reviewed Mar 10, 2025

View reviewed changes

Bryanskiy force-pushed the leak branch from 6c33a56 to a22ccdd Compare March 20, 2025 15:10

This comment has been minimized.

Sign in to view

Initial support for auto traits with default bounds

ad9504e

Bryanskiy force-pushed the leak branch from a22ccdd to ad9504e Compare March 20, 2025 15:29

Bryanskiy mentioned this pull request Mar 21, 2025

Tracking Issue for experiment with default auto traits #138781

Open

13 tasks

		/// For optimization purposes instead of adding default supertraits, bounds
		/// are added to the associative items:

Initial support for auto traits with default bounds #120706

Are you sure you want to change the base?

Initial support for auto traits with default bounds #120706

Conversation

Bryanskiy commented Feb 6, 2024 • edited Loading

Intro

Default bounds for old editions

Supertraits

Trait objects

Foreign types

Unresolved questions

Bryanskiy commented Feb 6, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

petrochenkov commented Feb 8, 2024 • edited Loading

petrochenkov commented Feb 8, 2024

This comment has been minimized.

Bryanskiy commented Feb 8, 2024 • edited Loading

lcnr commented Feb 8, 2024

Bryanskiy commented Aug 20, 2024

bors commented Aug 27, 2024

Ddystopia commented Feb 7, 2025 • edited Loading

zetanumbers commented Feb 7, 2025

zetanumbers commented Feb 7, 2025 • edited Loading

LFS6502 commented Mar 1, 2025

Bryanskiy commented Mar 1, 2025

Bryanskiy commented Mar 8, 2025

lcnr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lcnr Mar 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lcnr Mar 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Mar 12, 2025

This comment has been minimized.

Bryanskiy commented Feb 6, 2024 •

edited

Loading

petrochenkov commented Feb 8, 2024 •

edited

Loading

Bryanskiy commented Feb 8, 2024 •

edited

Loading

Ddystopia commented Feb 7, 2025 •

edited

Loading

zetanumbers commented Feb 7, 2025 •

edited

Loading

lcnr Mar 10, 2025 •

edited

Loading

lcnr Mar 10, 2025 •

edited

Loading