From a81ecc14bb25d79b24ea05571c7f51ac67414f10 Mon Sep 17 00:00:00 2001 From: Boxy Date: Mon, 1 Apr 2024 03:55:41 +0100 Subject: [PATCH 1/5] Initial rewrite --- src/param_env.md | 170 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 130 insertions(+), 40 deletions(-) diff --git a/src/param_env.md b/src/param_env.md index 08e19c339..0e4c25f67 100644 --- a/src/param_env.md +++ b/src/param_env.md @@ -1,69 +1,159 @@ -# Parameter Environment +# The `ParamEnv` type -When working with associated and/or generic items (types, constants, -functions/methods) it is often relevant to have more information about the -`Self` or generic parameters. Trait bounds and similar information is encoded in -the [`ParamEnv`][pe]. Often this is not enough information to obtain things like the -type's `Layout`, but you can do all kinds of other checks on it (e.g. whether a -type implements `Copy`) or you can evaluate an associated constant whose value -does not depend on anything from the parameter environment. +## What is it -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +The type system relies on information in the environment in order for it to function correctly. This information is stored in the [`ParamEnv`][pe] type and it is important to use the correct `ParamEnv` when interacting with the type system. -For example if you have a function +The information represented by `ParamEnv` is a list of in scope where clauses, and a [`Reveal`][reveal]. A `ParamEnv` typically corresponds to a specific item's environment however it can also be created with arbitrary data that is not derived from a specific item. In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. +If we have a function such as: ```rust -fn foo(t: T) { ... } +// `foo` would have a `ParamEnv` of: +// `[T: Sized, T: Trait, ::Assoc: Clone]` +fn foo() +where + ::Assoc: Clone, +{} ``` +If we were interacting with the type system from inside of `foo` we would use this `ParamEnv` everywhere that we interact with the type system. This would allow things such as normalization (TODO: write a chapter about normalization and link it), evaluating generic constants, and proving where clauses/goals, to rely on `T` being sized, implementing `Trait`, etc. -the parameter environment for that function is `[T: Copy]`. This means any -evaluation within this function will, when accessing the type `T`, know about -its `Copy` bound via the parameter environment. +A more concrete example: +```rust +// `foo` would have a `ParamEnv` of: +// `[T: Sized, T: Clone]` +fn foo(a: T) -> (T, T) { + // when typechecking `foo` we require all the where clauses on `bar` + // to hold in order for it to be legal to call. This means we have to + // prove `T: Clone`. As we are type checking `foo` we use `foo`'s + // environment when trying to check that `T: Clone` holds. + // + // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized, T: Clone]` + // will trivially succeed as bound we want to prove is in our environment. + requires_clone(a); +} +``` -You can get the parameter environment for a `def_id` using the -[`param_env`][query] query. However, this `ParamEnv` can be too generic for -your use case. Using the `ParamEnv` from the surrounding context can allow you -to evaluate more things. For example, suppose we had something the following: +Or alternatively an example that would not compile: +```rust +// `foo2` would have a `ParamEnv` of: +// `[T: Sized]` +fn foo2(a: T) -> (T, T) { + // When typechecking `foo2` we attempt to prove `T: Clone`. + // As we are type checking `foo2` we use `foo2`'s environment + // when trying to prove `T: Clone`. + // + // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized]` will + // fail as there is nothing in the environment telling the trait solver + // that `T` implements `Clone`. + requires_clone(a); +} +``` -[query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ty_utils/ty/fn.param_env.html +It's very important to use the correct `ParamEnv` when interacting with the type system as otherwise it can lead to ICEs or things compiling when they shouldn't (or vice versa). See [#82159](https://github.com/rust-lang/rust/pull/82159) and [#82067](https://github.com/rust-lang/rust/pull/82067) as examples of PRs that changed rustc to use the correct param env to avoid ICE. + +## Construction + +Creating a `ParamEnv` is more complicated than simply using the list of where clauses defined on an item as written by the user. We need to both elaborate supertraits into the env and fully normalize all aliases. This logic is handled by [`traits::normalize_param_env_or_error`][normalize_env_or_error] (even though it does not mention anything about elaboration). + +### Elaborating supertraits + +When we have a function such as `fn foo()` we would like to be able to prove `T: Clone` inside of the function as the `Copy` trait has a `Clone` supertrait. Constructing a `ParamEnv` looks at all of the trait bounds in the env and explicitly adds new where clauses to the `ParamEnv` for any supertraits found on the traits. + +A concrete example would be the following function: +```rust +trait Trait: SuperTrait {} +trait SuperTrait: SuperSuperTrait {} + +// `bar`'s unelaborated `ParamEnv` would be: +// `[T: Sized, T: Copy, T: Trait]` +fn bar(a: T) { + requires_impl(a); +} + +fn requires_impl(a: T) {} +``` + +If we did not elaborate the env then the `requires_impl` call would fail to typecheck as we would not be able to prove `T: Clone` or `T: SuperSuperTrait`. In practice we elaborate the env which means that `bar`'s `ParamEnv` is actually: +`[T: Sized, T: Copy, T: Clone, T: Trait, T: SuperTrait, T: SuperSuperTrait]` +This allows us to prove `T: Clone` and `T: SuperSuperTrait` when type checking `bar`. + +The `Clone` trait has a `Sized` supertrait however we do not end up with two `T: Sized` bounds in the env (one for the supertrait and one for the implicitly added `T: Sized` bound). This is because the elaboration process (implemented via [`util::elaborate`][elaborate]) deduplicates the where clauses to avoid this. +As a side effect this also means that even if no actual elaboration of supertraits takes place, the existing where clauses in the env are _also_ deduplicated. See the following example: ```rust -trait Foo { +trait Trait {} +// The unelaborated `ParamEnv` would be: +// `[T: Sized, T: Trait, T: Trait]` +// but after elaboration it would be: +// `[T: Sized, T: Trait]` +fn foo() {} +``` + +The [next-gen trait solver][next-gen-solver] also requires this elaboration to take place. + +[elaborate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/util/fn.elaborate.html +[next-gen-solver]: https://rustc-dev-guide.rust-lang.org/solve/trait-solving.html + +### Normalizing all bounds + +In the old trait solver the where clauses stored in `ParamEnv` are required to be fully normalized or else the trait solver will not function correctly. A concrete example of needing to normalize the `ParamEnv` is the following: +```rust +trait Trait { type Assoc; } -trait Bar { } +trait Other { + type Bar; +} -trait Baz { - fn stuff() -> bool; +impl Other for T { + type Bar = u32; } -fn foo(t: T) +// `foo`'s unnormalized `ParamEnv` would be: +// `[T: Sized, U: Sized, U: Trait]` +fn foo(a: U) where - T: Foo, - ::Assoc: Bar + U: Trait<::Bar>, { - bar::() + requires_impl(a); } -fn bar() { - if T::stuff() { mep() } else { mop() } -} +fn requires_impl>(_: U) {} ``` -We may know some things inside `bar` that we wouldn't know if we just fetched -`bar`'s param env because of the `::Assoc: Bar` bound in `foo`. This -is a contrived example that makes no sense in our existing analyses, but we may -run into similar cases when doing analyses with associated constants on generic -traits or traits with assoc types. +As humans we can tell that `::Bar` is equal to `u32` so the trait bound on `U` is equivalent to `U: Trait`. In practice trying to prove `U: Trait` in the old solver in this environment would fail as it is unable to determine that `::Bar` is equal to `u32`. + +To work around this we normalize `ParamEnv`'s after constructing them so that `foo`'s `ParamEnv` is actually: `[T: Sized, U: Sized, U: Trait]` which means the trait solver is now able to use the `U: Trait` in the `ParamEnv` to determine that the trait bound `U: Trait` holds. + +This workaround does not work in all cases as normalizing associated types requires a `ParamEnv` which introduces a bootstrapping problem. We need a normalized `ParamEnv` in order for normalization to give correct results, but we need to normalize to get that `ParamEnv`. Currently we normalize the `ParamEnv` once using the unnormalized param env and it tends to give okay results in practice even though there are some examples where this breaks ([example]). + +In the next-gen trait solver the requirement for all where clauses in the `ParamEnv` to be fully normalized is not present and so we do not normalize when constructing `ParamEnv`s. + +[example]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e6933265ea3e84eaa47019465739992c + +--- + +When needing a `ParamEnv` in the compiler there generally three options for obtaining one: +- The correct env is already in scope simply use it (or pass it down the call stack to where you are) +- Call `tcx.param_env(def_id)` +- Use [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then call `traits::normalize_param_env_or_error` which will handle normalizing and elaborating all the where clauses in the env for you. + +In the large majority of cases a `ParamEnv` when required already exists somewhere in scope or above in the call stack and should be passed down. + +Using the `param_env` query to obtain an env is generally done at the start of some kind of analysis and then passed everywhere that a `ParamEnv` is required. For example the type checker will create a `ParamEnv` for the item it is type checking and then pass it around everywhere. + +Creating an env from an arbitrary set of where clauses is usually unnecessary and should only be done if the environment you need does not correspond to an actual item in the source code. + +[param_env_new]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.new +[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html ## Bundling -Another great thing about `ParamEnv` is that you can use it to bundle the thing -depending on generic parameters (e.g. a `Ty`) by calling the [`and`][and] -method. This will produce a [`ParamEnvAnd`][pea], making clear that you -should probably not be using the inner value without taking care to also use -the [`ParamEnv`][pe]. +A useful API on `ParamEnv` is the [`and`][and] method which allows bundling a value with the `ParamEnv`. The `and` method produces a [`ParamEnvAnd`][pea] making it clearer that using the inner value is intended to be done in that specific environment. [and]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.and [pea]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnvAnd.html +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ty_utils/ty/fn.param_env.html +[reveal]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/enum.Reveal.html \ No newline at end of file From 4da1ebbf23655881c45872c91ec4a53a3e3e0940 Mon Sep 17 00:00:00 2001 From: Boxy Date: Mon, 1 Apr 2024 04:01:28 +0100 Subject: [PATCH 2/5] Change location in guide --- src/SUMMARY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index a4be4b929..d8f638f0e 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -117,6 +117,7 @@ - [Generic arguments](./generic_arguments.md) - [Constants in the type system](./constants.md) - [Bound vars and Parameters](./bound-vars-and-params.md) +- [Parameter Environments](./param_env.md) - [Type inference](./type-inference.md) - [Trait solving](./traits/resolution.md) - [Early and Late Bound Parameter Definitions](./early-late-bound-summary.md) @@ -167,7 +168,6 @@ - [Closure constraints](./borrow_check/region_inference/closure_constraints.md) - [Error reporting](./borrow_check/region_inference/error_reporting.md) - [Two-phase-borrows](./borrow_check/two_phase_borrows.md) -- [Parameter Environments](./param_env.md) - [Errors and Lints](diagnostics.md) - [Diagnostic and subdiagnostic structs](./diagnostics/diagnostic-structs.md) - [Translation](./diagnostics/translation.md) From 8bdfd62681ac6ef13b884d7b02af3f327b6b23d3 Mon Sep 17 00:00:00 2001 From: Boxy Date: Mon, 1 Apr 2024 04:34:21 +0100 Subject: [PATCH 3/5] reviews --- src/param_env.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/param_env.md b/src/param_env.md index 0e4c25f67..fbf95aa5d 100644 --- a/src/param_env.md +++ b/src/param_env.md @@ -4,7 +4,7 @@ The type system relies on information in the environment in order for it to function correctly. This information is stored in the [`ParamEnv`][pe] type and it is important to use the correct `ParamEnv` when interacting with the type system. -The information represented by `ParamEnv` is a list of in scope where clauses, and a [`Reveal`][reveal]. A `ParamEnv` typically corresponds to a specific item's environment however it can also be created with arbitrary data that is not derived from a specific item. In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. +The information represented by `ParamEnv` is a list of in-scope where-clauses, and a [`Reveal`][reveal]. A `ParamEnv` typically corresponds to a specific item's environment however it can also be created with arbitrary data that is not derived from a specific item. In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. If we have a function such as: ```rust @@ -21,7 +21,7 @@ A more concrete example: ```rust // `foo` would have a `ParamEnv` of: // `[T: Sized, T: Clone]` -fn foo(a: T) -> (T, T) { +fn foo(a: T) { // when typechecking `foo` we require all the where clauses on `bar` // to hold in order for it to be legal to call. This means we have to // prove `T: Clone`. As we are type checking `foo` we use `foo`'s @@ -37,14 +37,15 @@ Or alternatively an example that would not compile: ```rust // `foo2` would have a `ParamEnv` of: // `[T: Sized]` -fn foo2(a: T) -> (T, T) { +fn foo2(a: T) { // When typechecking `foo2` we attempt to prove `T: Clone`. // As we are type checking `foo2` we use `foo2`'s environment // when trying to prove `T: Clone`. // // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized]` will // fail as there is nothing in the environment telling the trait solver - // that `T` implements `Clone`. + // that `T` implements `Clone` and there exists no user written impl + // that could apply. requires_clone(a); } ``` From 0817132f5b8a6a5a6302b7c98d16e030989da35f Mon Sep 17 00:00:00 2001 From: Boxy Date: Mon, 1 Apr 2024 17:17:25 +0100 Subject: [PATCH 4/5] Reorganise into subchapters and explain `Reveal` --- src/SUMMARY.md | 5 +- src/param_env.md | 160 ------------------ src/param_env/param_env_acquisition.md | 43 +++++ .../param_env_construction_internals.md | 82 +++++++++ src/param_env/param_env_summary.md | 18 ++ src/param_env/param_env_what_is_it.md | 60 +++++++ 6 files changed, 207 insertions(+), 161 deletions(-) delete mode 100644 src/param_env.md create mode 100644 src/param_env/param_env_acquisition.md create mode 100644 src/param_env/param_env_construction_internals.md create mode 100644 src/param_env/param_env_summary.md create mode 100644 src/param_env/param_env_what_is_it.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index d8f638f0e..343a704dc 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -117,7 +117,10 @@ - [Generic arguments](./generic_arguments.md) - [Constants in the type system](./constants.md) - [Bound vars and Parameters](./bound-vars-and-params.md) -- [Parameter Environments](./param_env.md) +- [Parameter Environments](./param_env/param_env_summary.md) + - [What is it?](./param_env/param_env_what_is_it.md) + - [How are `ParamEnv`'s constructed internally](./param_env/param_env_construction_internals.md) + - [Which `ParamEnv` do I use?](./param_env/param_env_acquisition.md) - [Type inference](./type-inference.md) - [Trait solving](./traits/resolution.md) - [Early and Late Bound Parameter Definitions](./early-late-bound-summary.md) diff --git a/src/param_env.md b/src/param_env.md deleted file mode 100644 index fbf95aa5d..000000000 --- a/src/param_env.md +++ /dev/null @@ -1,160 +0,0 @@ -# The `ParamEnv` type - -## What is it - -The type system relies on information in the environment in order for it to function correctly. This information is stored in the [`ParamEnv`][pe] type and it is important to use the correct `ParamEnv` when interacting with the type system. - -The information represented by `ParamEnv` is a list of in-scope where-clauses, and a [`Reveal`][reveal]. A `ParamEnv` typically corresponds to a specific item's environment however it can also be created with arbitrary data that is not derived from a specific item. In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. - -If we have a function such as: -```rust -// `foo` would have a `ParamEnv` of: -// `[T: Sized, T: Trait, ::Assoc: Clone]` -fn foo() -where - ::Assoc: Clone, -{} -``` -If we were interacting with the type system from inside of `foo` we would use this `ParamEnv` everywhere that we interact with the type system. This would allow things such as normalization (TODO: write a chapter about normalization and link it), evaluating generic constants, and proving where clauses/goals, to rely on `T` being sized, implementing `Trait`, etc. - -A more concrete example: -```rust -// `foo` would have a `ParamEnv` of: -// `[T: Sized, T: Clone]` -fn foo(a: T) { - // when typechecking `foo` we require all the where clauses on `bar` - // to hold in order for it to be legal to call. This means we have to - // prove `T: Clone`. As we are type checking `foo` we use `foo`'s - // environment when trying to check that `T: Clone` holds. - // - // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized, T: Clone]` - // will trivially succeed as bound we want to prove is in our environment. - requires_clone(a); -} -``` - -Or alternatively an example that would not compile: -```rust -// `foo2` would have a `ParamEnv` of: -// `[T: Sized]` -fn foo2(a: T) { - // When typechecking `foo2` we attempt to prove `T: Clone`. - // As we are type checking `foo2` we use `foo2`'s environment - // when trying to prove `T: Clone`. - // - // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized]` will - // fail as there is nothing in the environment telling the trait solver - // that `T` implements `Clone` and there exists no user written impl - // that could apply. - requires_clone(a); -} -``` - -It's very important to use the correct `ParamEnv` when interacting with the type system as otherwise it can lead to ICEs or things compiling when they shouldn't (or vice versa). See [#82159](https://github.com/rust-lang/rust/pull/82159) and [#82067](https://github.com/rust-lang/rust/pull/82067) as examples of PRs that changed rustc to use the correct param env to avoid ICE. - -## Construction - -Creating a `ParamEnv` is more complicated than simply using the list of where clauses defined on an item as written by the user. We need to both elaborate supertraits into the env and fully normalize all aliases. This logic is handled by [`traits::normalize_param_env_or_error`][normalize_env_or_error] (even though it does not mention anything about elaboration). - -### Elaborating supertraits - -When we have a function such as `fn foo()` we would like to be able to prove `T: Clone` inside of the function as the `Copy` trait has a `Clone` supertrait. Constructing a `ParamEnv` looks at all of the trait bounds in the env and explicitly adds new where clauses to the `ParamEnv` for any supertraits found on the traits. - -A concrete example would be the following function: -```rust -trait Trait: SuperTrait {} -trait SuperTrait: SuperSuperTrait {} - -// `bar`'s unelaborated `ParamEnv` would be: -// `[T: Sized, T: Copy, T: Trait]` -fn bar(a: T) { - requires_impl(a); -} - -fn requires_impl(a: T) {} -``` - -If we did not elaborate the env then the `requires_impl` call would fail to typecheck as we would not be able to prove `T: Clone` or `T: SuperSuperTrait`. In practice we elaborate the env which means that `bar`'s `ParamEnv` is actually: -`[T: Sized, T: Copy, T: Clone, T: Trait, T: SuperTrait, T: SuperSuperTrait]` -This allows us to prove `T: Clone` and `T: SuperSuperTrait` when type checking `bar`. - -The `Clone` trait has a `Sized` supertrait however we do not end up with two `T: Sized` bounds in the env (one for the supertrait and one for the implicitly added `T: Sized` bound). This is because the elaboration process (implemented via [`util::elaborate`][elaborate]) deduplicates the where clauses to avoid this. - -As a side effect this also means that even if no actual elaboration of supertraits takes place, the existing where clauses in the env are _also_ deduplicated. See the following example: -```rust -trait Trait {} -// The unelaborated `ParamEnv` would be: -// `[T: Sized, T: Trait, T: Trait]` -// but after elaboration it would be: -// `[T: Sized, T: Trait]` -fn foo() {} -``` - -The [next-gen trait solver][next-gen-solver] also requires this elaboration to take place. - -[elaborate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/util/fn.elaborate.html -[next-gen-solver]: https://rustc-dev-guide.rust-lang.org/solve/trait-solving.html - -### Normalizing all bounds - -In the old trait solver the where clauses stored in `ParamEnv` are required to be fully normalized or else the trait solver will not function correctly. A concrete example of needing to normalize the `ParamEnv` is the following: -```rust -trait Trait { - type Assoc; -} - -trait Other { - type Bar; -} - -impl Other for T { - type Bar = u32; -} - -// `foo`'s unnormalized `ParamEnv` would be: -// `[T: Sized, U: Sized, U: Trait]` -fn foo(a: U) -where - U: Trait<::Bar>, -{ - requires_impl(a); -} - -fn requires_impl>(_: U) {} -``` - -As humans we can tell that `::Bar` is equal to `u32` so the trait bound on `U` is equivalent to `U: Trait`. In practice trying to prove `U: Trait` in the old solver in this environment would fail as it is unable to determine that `::Bar` is equal to `u32`. - -To work around this we normalize `ParamEnv`'s after constructing them so that `foo`'s `ParamEnv` is actually: `[T: Sized, U: Sized, U: Trait]` which means the trait solver is now able to use the `U: Trait` in the `ParamEnv` to determine that the trait bound `U: Trait` holds. - -This workaround does not work in all cases as normalizing associated types requires a `ParamEnv` which introduces a bootstrapping problem. We need a normalized `ParamEnv` in order for normalization to give correct results, but we need to normalize to get that `ParamEnv`. Currently we normalize the `ParamEnv` once using the unnormalized param env and it tends to give okay results in practice even though there are some examples where this breaks ([example]). - -In the next-gen trait solver the requirement for all where clauses in the `ParamEnv` to be fully normalized is not present and so we do not normalize when constructing `ParamEnv`s. - -[example]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e6933265ea3e84eaa47019465739992c - ---- - -When needing a `ParamEnv` in the compiler there generally three options for obtaining one: -- The correct env is already in scope simply use it (or pass it down the call stack to where you are) -- Call `tcx.param_env(def_id)` -- Use [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then call `traits::normalize_param_env_or_error` which will handle normalizing and elaborating all the where clauses in the env for you. - -In the large majority of cases a `ParamEnv` when required already exists somewhere in scope or above in the call stack and should be passed down. - -Using the `param_env` query to obtain an env is generally done at the start of some kind of analysis and then passed everywhere that a `ParamEnv` is required. For example the type checker will create a `ParamEnv` for the item it is type checking and then pass it around everywhere. - -Creating an env from an arbitrary set of where clauses is usually unnecessary and should only be done if the environment you need does not correspond to an actual item in the source code. - -[param_env_new]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.new -[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html - -## Bundling - -A useful API on `ParamEnv` is the [`and`][and] method which allows bundling a value with the `ParamEnv`. The `and` method produces a [`ParamEnvAnd`][pea] making it clearer that using the inner value is intended to be done in that specific environment. - -[and]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.and -[pea]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnvAnd.html -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html -[query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ty_utils/ty/fn.param_env.html -[reveal]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/enum.Reveal.html \ No newline at end of file diff --git a/src/param_env/param_env_acquisition.md b/src/param_env/param_env_acquisition.md new file mode 100644 index 000000000..1289f133a --- /dev/null +++ b/src/param_env/param_env_acquisition.md @@ -0,0 +1,43 @@ + +# Which `ParamEnv` do I use? + +When needing a [`ParamEnv`][pe] in the compiler there are a few options for obtaining one: +- The correct env is already in scope simply use it (or pass it down the call stack to where you are). +- The [`tcx.param_env(def_id)` query][param_env_query] +- Use [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then call `traits::normalize_param_env_or_error` which will handle normalizing and elaborating all the where clauses in the env for you. +- Creating an empty environment via [`ParamEnv::reveal_all`][env_reveal_all] or [`ParamEnv::empty`][env_empty] + +In the large majority of cases a `ParamEnv` when required already exists somewhere in scope or above in the call stack and should be passed down. A non exhaustive list of places where you might find an existing `ParamEnv`: +- During typeck `FnCtxt` has a [`param_env` field][fnctxt_param_env] +- When writing late lints the `LateContext` has a [`param_env` field][latectxt_param_env] +- During well formedness checking the `WfCheckingCtxt` has a [`param_env` field][wfckctxt_param_env] +- The `TypeChecker` used by Mir Typeck has a [`param_env` field][mirtypeck_param_env] +- In the next-gen trait solver all `Goal`s have a [`param_env` field][goal_param_env] specifying what environment to prove the goal in +- When editing an existing [`TypeRelation`][typerelation] if it implements `ObligationEmittingRelation` then a [`param_env` method][typerelation_param_env] will be available. + +Using the `param_env` query to obtain an env is generally done at the start of some kind of analysis and then passed everywhere that a `ParamEnv` is required. For example the type checker will create a `ParamEnv` for the item it is type checking and then pass it around everywhere. + +Creating an env from an arbitrary set of where clauses is usually unnecessary and should only be done if the environment you need does not correspond to an actual item in the source code (i.e. [`compare_method_predicate_entailment`][method_pred_entailment] as mentioned earlier). + +Creating an empty environment via `ParamEnv::empty` is almost always wrong. There are very few places where we actually know that the environment should be empty. One of the only places where we do actually know this is after monomorphization, however the `ParamEnv` there should be constructed via `ParamEnv::reveal_all` instead as at this point we should be able to determine the hidden type of opaque types. Codegen/Post-mono is one of the only places that should be using `ParamEnv::reveal_all`. + +An additional piece of complexity here is specifying the [`Reveal`][reveal] (see linked docs for explanation of what reveal does) used for the `ParamEnv`. When constructing a param env using the `param_env` query it will have `Reveal::UserFacing`, if `Reveal::All` is desired then the [`tcx.param_env_reveal_all_normalized`][env_reveal_all_normalized] query can be used instead. + +The `ParamEnv` type has a method [`ParamEnv::with_reveal_all_normalized`][with_reveal_all] which converts an existing `ParamEnv` into one with `Reveal::All` specified. Where possible the previously mentioned query should be preferred as it is more efficient. + +[param_env_new]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.new +[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html +[fnctxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/struct.FnCtxt.html#structfield.param_env +[latectxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/context/struct.LateContext.html#structfield.param_env +[wfckctxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/wfcheck/struct.WfCheckingCtxt.html#structfield.param_env +[goal_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/solve/struct.Goal.html#structfield.param_env +[typerelation_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/infer/relate/combine/trait.ObligationEmittingRelation.html#tymethod.param_env +[typerelation]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/relate/trait.TypeRelation.html +[mirtypeck_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_borrowck/type_check/struct.TypeChecker.html#structfield.param_env +[env_reveal_all_normalized]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env_reveal_all_normalized +[with_reveal_all]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.with_reveal_all_normalized +[env_reveal_all]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.reveal_all +[env_empty]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.empty +[reveal]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/enum.Reveal.html +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[param_env_query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env \ No newline at end of file diff --git a/src/param_env/param_env_construction_internals.md b/src/param_env/param_env_construction_internals.md new file mode 100644 index 000000000..ef7537bc8 --- /dev/null +++ b/src/param_env/param_env_construction_internals.md @@ -0,0 +1,82 @@ + +# How are `ParamEnv`'s constructed internally? + +Creating a [`ParamEnv`][pe] is more complicated than simply using the list of where clauses defined on an item as written by the user. We need to both elaborate supertraits into the env and fully normalize all aliases. This logic is handled by [`traits::normalize_param_env_or_error`][normalize_env_or_error] (even though it does not mention anything about elaboration). + +## Elaborating supertraits + +When we have a function such as `fn foo()` we would like to be able to prove `T: Clone` inside of the function as the `Copy` trait has a `Clone` supertrait. Constructing a `ParamEnv` looks at all of the trait bounds in the env and explicitly adds new where clauses to the `ParamEnv` for any supertraits found on the traits. + +A concrete example would be the following function: +```rust +trait Trait: SuperTrait {} +trait SuperTrait: SuperSuperTrait {} + +// `bar`'s unelaborated `ParamEnv` would be: +// `[T: Sized, T: Copy, T: Trait]` +fn bar(a: T) { + requires_impl(a); +} + +fn requires_impl(a: T) {} +``` + +If we did not elaborate the env then the `requires_impl` call would fail to typecheck as we would not be able to prove `T: Clone` or `T: SuperSuperTrait`. In practice we elaborate the env which means that `bar`'s `ParamEnv` is actually: +`[T: Sized, T: Copy, T: Clone, T: Trait, T: SuperTrait, T: SuperSuperTrait]` +This allows us to prove `T: Clone` and `T: SuperSuperTrait` when type checking `bar`. + +The `Clone` trait has a `Sized` supertrait however we do not end up with two `T: Sized` bounds in the env (one for the supertrait and one for the implicitly added `T: Sized` bound). This is because the elaboration process (implemented via [`util::elaborate`][elaborate]) deduplicates the where clauses to avoid this. + +As a side effect this also means that even if no actual elaboration of supertraits takes place, the existing where clauses in the env are _also_ deduplicated. See the following example: +```rust +trait Trait {} +// The unelaborated `ParamEnv` would be: +// `[T: Sized, T: Trait, T: Trait]` +// but after elaboration it would be: +// `[T: Sized, T: Trait]` +fn foo() {} +``` + +The [next-gen trait solver][next-gen-solver] also requires this elaboration to take place. + +[elaborate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/util/fn.elaborate.html +[next-gen-solver]: https://rustc-dev-guide.rust-lang.org/solve/trait-solving.html + +## Normalizing all bounds + +In the old trait solver the where clauses stored in `ParamEnv` are required to be fully normalized or else the trait solver will not function correctly. A concrete example of needing to normalize the `ParamEnv` is the following: +```rust +trait Trait { + type Assoc; +} + +trait Other { + type Bar; +} + +impl Other for T { + type Bar = u32; +} + +// `foo`'s unnormalized `ParamEnv` would be: +// `[T: Sized, U: Sized, U: Trait]` +fn foo(a: U) +where + U: Trait<::Bar>, +{ + requires_impl(a); +} + +fn requires_impl>(_: U) {} +``` + +As humans we can tell that `::Bar` is equal to `u32` so the trait bound on `U` is equivalent to `U: Trait`. In practice trying to prove `U: Trait` in the old solver in this environment would fail as it is unable to determine that `::Bar` is equal to `u32`. + +To work around this we normalize `ParamEnv`'s after constructing them so that `foo`'s `ParamEnv` is actually: `[T: Sized, U: Sized, U: Trait]` which means the trait solver is now able to use the `U: Trait` in the `ParamEnv` to determine that the trait bound `U: Trait` holds. + +This workaround does not work in all cases as normalizing associated types requires a `ParamEnv` which introduces a bootstrapping problem. We need a normalized `ParamEnv` in order for normalization to give correct results, but we need to normalize to get that `ParamEnv`. Currently we normalize the `ParamEnv` once using the unnormalized param env and it tends to give okay results in practice even though there are some examples where this breaks ([example]). + +In the next-gen trait solver the requirement for all where clauses in the `ParamEnv` to be fully normalized is not present and so we do not normalize when constructing `ParamEnv`s. + +[example]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e6933265ea3e84eaa47019465739992c +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html \ No newline at end of file diff --git a/src/param_env/param_env_summary.md b/src/param_env/param_env_summary.md new file mode 100644 index 000000000..0ff6d8fc3 --- /dev/null +++ b/src/param_env/param_env_summary.md @@ -0,0 +1,18 @@ +# The `ParamEnv` type + +## Summary + +The [`ParamEnv`][pe] is used to store information about the environment that we are interacting with the type system from. For example the set of in-scope where-clauses is stored in `ParamEnv` as it differs between each item whereas the list of user written impls is not stored in the `ParamEnv` as this does not change for each item. + +This chapter of the dev guide covers: +- A high level summary of what a `ParamEnv` is and what it is used for +- Technical details about what the process of constructing a `ParamEnv` involves +- Guidance about how to acquire a `ParamEnv` when one is required + +## Bundling + +A useful API on `ParamEnv` is the [`and`][and] method which allows bundling a value with the `ParamEnv`. The `and` method produces a [`ParamEnvAnd`][pea] making it clearer that using the inner value is intended to be done in that specific environment. + +[and]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.and +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[pea]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnvAnd.html \ No newline at end of file diff --git a/src/param_env/param_env_what_is_it.md b/src/param_env/param_env_what_is_it.md new file mode 100644 index 000000000..ca09518d9 --- /dev/null +++ b/src/param_env/param_env_what_is_it.md @@ -0,0 +1,60 @@ + +# What is a `ParamEnv`? + +The type system relies on information in the environment in order for it to function correctly. This information is stored in the [`ParamEnv`][pe] type and it is important to use the correct `ParamEnv` when interacting with the type system. + +The information represented by `ParamEnv` is a list of in-scope where-clauses, and a [`Reveal`][reveal] (see linked docs for more information). A `ParamEnv` typically corresponds to a specific item's where clauses, some clauses are not explicitly written bounds and instead are implicitly added in [`predicates_of`][predicates_of] such as `ConstArgHasType` or some implied bounds. + +A `ParamEnv` can also be created with arbitrary data that is not derived from a specific item such as in [`compare_method_predicate_entailment`][method_pred_entailment] which creates a hybrid `ParamEnv` consisting of the impl's where clauses and the trait definition's function's where clauses. In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. + +If we have a function such as: +```rust +// `foo` would have a `ParamEnv` of: +// `[T: Sized, T: Trait, ::Assoc: Clone]` +fn foo() +where + ::Assoc: Clone, +{} +``` +If we were conceptually inside of `foo` (for example, type-checking or linting it) we would use this `ParamEnv` everywhere that we interact with the type system. This would allow things such as normalization (TODO: write a chapter about normalization and link it), evaluating generic constants, and proving where clauses/goals, to rely on `T` being sized, implementing `Trait`, etc. + +A more concrete example: +```rust +// `foo` would have a `ParamEnv` of: +// `[T: Sized, T: Clone]` +fn foo(a: T) { + // when typechecking `foo` we require all the where clauses on `bar` + // to hold in order for it to be legal to call. This means we have to + // prove `T: Clone`. As we are type checking `foo` we use `foo`'s + // environment when trying to check that `T: Clone` holds. + // + // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized, T: Clone]` + // will trivially succeed as bound we want to prove is in our environment. + requires_clone(a); +} +``` + +Or alternatively an example that would not compile: +```rust +// `foo2` would have a `ParamEnv` of: +// `[T: Sized]` +fn foo2(a: T) { + // When typechecking `foo2` we attempt to prove `T: Clone`. + // As we are type checking `foo2` we use `foo2`'s environment + // when trying to prove `T: Clone`. + // + // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized]` will + // fail as there is nothing in the environment telling the trait solver + // that `T` implements `Clone` and there exists no user written impl + // that could apply. + requires_clone(a); +} +``` + +It's very important to use the correct `ParamEnv` when interacting with the type system as otherwise it can lead to ICEs or things compiling when they shouldn't (or vice versa). See [#82159](https://github.com/rust-lang/rust/pull/82159) and [#82067](https://github.com/rust-lang/rust/pull/82067) as examples of PRs that changed rustc to use the correct param env to avoid ICE. Determining how to acquire the correct `ParamEnv` is explained later in this chapter. + +[predicates_of]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/collect/predicates_of/fn.predicates_of.html +[method_pred_entailment]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env +[reveal]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/enum.Reveal.html \ No newline at end of file From 4f3318efdae3822eac8c6cad4edd9d0e861ff97a Mon Sep 17 00:00:00 2001 From: Boxy Date: Mon, 1 Apr 2024 17:24:45 +0100 Subject: [PATCH 5/5] Fix some broken links --- src/param_env/param_env_acquisition.md | 5 +++-- src/param_env/param_env_construction_internals.md | 3 ++- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/param_env/param_env_acquisition.md b/src/param_env/param_env_acquisition.md index 1289f133a..744186635 100644 --- a/src/param_env/param_env_acquisition.md +++ b/src/param_env/param_env_acquisition.md @@ -4,7 +4,7 @@ When needing a [`ParamEnv`][pe] in the compiler there are a few options for obtaining one: - The correct env is already in scope simply use it (or pass it down the call stack to where you are). - The [`tcx.param_env(def_id)` query][param_env_query] -- Use [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then call `traits::normalize_param_env_or_error` which will handle normalizing and elaborating all the where clauses in the env for you. +- Use [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then call [`traits::normalize_param_env_or_error`][normalize_env_or_error] which will handle normalizing and elaborating all the where clauses in the env for you. - Creating an empty environment via [`ParamEnv::reveal_all`][env_reveal_all] or [`ParamEnv::empty`][env_empty] In the large majority of cases a `ParamEnv` when required already exists somewhere in scope or above in the call stack and should be passed down. A non exhaustive list of places where you might find an existing `ParamEnv`: @@ -40,4 +40,5 @@ The `ParamEnv` type has a method [`ParamEnv::with_reveal_all_normalized`][with_r [env_empty]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.empty [reveal]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/enum.Reveal.html [pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html -[param_env_query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env \ No newline at end of file +[param_env_query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env +[method_pred_entailment]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html \ No newline at end of file diff --git a/src/param_env/param_env_construction_internals.md b/src/param_env/param_env_construction_internals.md index ef7537bc8..fa9744fc0 100644 --- a/src/param_env/param_env_construction_internals.md +++ b/src/param_env/param_env_construction_internals.md @@ -79,4 +79,5 @@ This workaround does not work in all cases as normalizing associated types requi In the next-gen trait solver the requirement for all where clauses in the `ParamEnv` to be fully normalized is not present and so we do not normalize when constructing `ParamEnv`s. [example]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e6933265ea3e84eaa47019465739992c -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html \ No newline at end of file +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html \ No newline at end of file