Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement impl Trait in return type position by anonymization. #35091

Merged
merged 9 commits into from
Aug 12, 2016

Conversation

eddyb
Copy link
Member

@eddyb eddyb commented Jul 28, 2016

This is the first step towards implementing impl Trait (cc #34511).
impl Trait types are only allowed in function and inherent method return types, and capture all named lifetime and type parameters, being invariant over them.
No lifetimes that are not explicitly named lifetime parameters are allowed to escape from the function body.
The exposed traits are only those listed explicitly, i.e. Foo and Clone in impl Foo + Clone, with the exception of "auto traits" (like Send or Sync) which "leak" the actual contents.

The implementation strategy is anonymization, i.e.:

fn foo<T>(xs: Vec<T>) -> impl Iterator<Item=impl FnOnce() -> T> {
    xs.into_iter().map(|x| || x)
}

// is represented as:
type A</*invariant over*/ T> where A<T>: Iterator<Item=B<T>>;
type B</*invariant over*/ T> where B<T>: FnOnce() -> T;
fn foo<T>(xs: Vec<T>) -> A<T> {
    xs.into_iter().map(|x| || x): $0 where $0: Iterator<Item=$1>, $1: FnOnce() -> T
}

$0 and $1 are resolved (to iter::Map<vec::Iter<T>, closure> and the closure, respectively) and assigned to A and B, after checking the body of foo. A and B are never resolved for user-facing type equality (typeck), but always for the low-level representation and specialization (trans).

The "auto traits" exception is implemented by collecting bounds like impl Trait: Send that have failed for the obscure impl Trait type (i.e. A or B above), pretending they succeeded within the function and trying them again after type-checking the whole crate, by replacing impl Trait with the real type.

While passing around values which have explicit lifetime parameters (of the function with -> impl Trait) in their type should work, regionck appears to assign inference variables in way too many cases, and never properly resolving them to either explicit lifetime parameters, or 'static.
We might not be able to handle lifetime parameters in impl Trait without changes to lifetime inference, but type parameters can have arbitrary lifetimes in them from the caller, so most type-generic usecases (or not generic at all) should not run into this problem.

cc @rust-lang/lang

@rust-highfive
Copy link
Collaborator

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

@Ryman
Copy link
Contributor

Ryman commented Jul 28, 2016

$0 and $1 are resolved (to iter::Mapvec::Iter<T, closure> and the closure, respectively) and assigned to A and B, after checking the body of foo.

Will the #[must_use] attribute still cause a warning if the return value is unused in this case?

@eddyb
Copy link
Member Author

eddyb commented Jul 28, 2016

@Ryman Not unless the type is normalized with Reveal::All in the lint, no.

@bors
Copy link
Contributor

bors commented Aug 1, 2016

☔ The latest upstream changes (presumably #35143) made this pull request unmergeable. Please resolve the merge conflicts.

@eddyb eddyb force-pushed the impl-trait branch 5 times, most recently from 506c6ff to 6a3ee28 Compare August 4, 2016 16:25
@@ -174,6 +174,7 @@ impl<'a, 'gcx, 'tcx> InferCtxt<'a, 'gcx, 'tcx> {
ty::TyEnum(..) | // OutlivesNominalType
ty::TyStruct(..) | // OutlivesNominalType
ty::TyBox(..) | // OutlivesNominalType (ish)
ty::TyAnon(..) | // OutlivesNominalType (ish)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment seems wrong. Perhaps "OutlivesProjectionComponents (ish)" -- really this merits a revising of the RFC 1214 rules to include OutlivesAnonType or something, but that's ok.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably this code is doing the right thing, but it's a bit more conservative than it might otherwise be. Really there are choices here just like for projections. Consider for example if you had trait Foo<'a,'b>: 'a, then you could in theory derive that impl Foo<'a,'b>: 'a even if 'b: 'a doesn't necessarily hold. Something similar could occur with trait Foo<'a, T>: 'a, where impl Foo<'a, &'b T>: 'a might hold even if 'b: 'a does not hold.

It seems in a way like an odd rule, since the type name that the type system gives includes lifetimes (like 'b) that are not necessarily in scope, but the same thing can occur with projections -- the point is that, after monomorphization, the normalized form of that type is known to outlive 'a, even if all components pre-normalization do not.

/me has that itchy feeling of wanting to revisit a richer, formalized version of this, to try and have a good reference. But nothing here makes me too scared yet. =)

@bors
Copy link
Contributor

bors commented Aug 6, 2016

☔ The latest upstream changes (presumably #35267) made this pull request unmergeable. Please resolve the merge conflicts.

@arielb1
Copy link
Contributor

arielb1 commented Aug 7, 2016

I don't like the "capture all lifetimes in the impl trait" approach and would prefer something more like lifetime elision.

@bors
Copy link
Contributor

bors commented Aug 9, 2016

☔ The latest upstream changes (presumably #35166) made this pull request unmergeable. Please resolve the merge conflicts.

/// scope, that anonymized types will close over. This property is
/// controlled by the region scope because it's fine-grained enough
/// to allow restriction of anonymized types to the syntactical extent
/// of a function's return type.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I can't make heads or tails of this comment :) An example would be great

Copy link
Member Author

@eddyb eddyb Aug 10, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you talking about allowing impl Trait, or the "generics in scope"? The latter would be easy to provide an example from (<'a, T> for fn foo<'a, T>(...) -> impl Trait), but controlling whether impl Trait is legal in a certain position is harder to explain.
For it to make more sense, rscope would have to become one of several properties that get propagated recursively.

At the same time, the ItemContext in typeck::collect may be a better choice now for holding the "generics in scope", as it's the only AstConv implementation which allows impl Trait, but it's still not that uniform (only the return type of a function or inherent impl method supports it, not just any type in the signature of those items).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eddyb after I read below I figured out what the comment meant. I think something like this would have helped me:


If this scope allows anonymized types, return the generics in scope that anonymized types will close over. For example, if you have a function like:

fn foo<'a, T>() -> impl Trait { ... }

then, for the scope that is used when handling the return type, this function would return 'a and T. This property is controlled by the region scope because it's fine-grained enough to allow restriction of anonymized types to the syntactical extentof a function's return type.

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Aug 10, 2016

I went through the RFC and made a minimal list of test cases I think would like to see:

  • positive tests for auto trait inference (something where we need to know that impl Trait: Send for the code to work)
  • negative tests for auto trait inference (something where the code requires that impl Trait: Send but in fact it does not)
  • test for interaction of auto trait inference and specialization (as mentioned in the comments)
  • negative test where choice of regions is ambiguous, e.g., as described in this comment in the final example was one where the code ought to fail
  • test where there IS a correct choice of regions (e.g. from that same comment); this could be positive or negative, depending on how the code currently behaves
  • negative tests for attempts to use impl Trait syntax in places where it is not permitted:
    • arguments
    • traits and impls of traits
    • impl header
    • fn() -> impl Foo or Fn() -> impl Foo types
    • where clauses
  • negative tests where we try to return values of two distinct types
  • negative test for recursive fn that uses impl trait as in this example
  • negative test that shows "true type" is not revealed to the caller (let x: u32 = foo())
  • positive test that shows that using specialization (or sizeof) you can "reveal" the true type -- but only at trans time
  • negative test shows an attempt to "leak type" with specialization fails
    • e.g., trait Foo { type T; } impl<T> Foo for T { default type T = (); } impl Foo for i32 { default type T = i32; } and then some code that would require <X as Foo>::T = i32 where X is an impl trait type
  • positive test for return type identity (two calls w/ same arguments yield "equal" types)
  • negative test for return type identity (two calls w/ a different type argument yield "unequal" types)
  • optional: test for region return type identity and region inference (because the two types must be the same, a loan is extended?)
  • feature-gate testing (uses of impl trait where feature gate is NOT enabled)

@nikomatsakis
Copy link
Contributor

Er, I posted this comment earlier, but in the wrong place:

@eddyb ok so as discussed on IRC, I am in favor of landing this PR, on the assumption that we will address some of the thornier issues later.

One thing I think is missing is negative tests. I'd like to see at least some tests that if you try to return incompatible types that will result in an error. Probably some (more?) tests targeting "Auto trait" inference -- and the limits, e.g. around specialization -- are warranted as well.

One thing I have done in the past -- but didn't do yet for this PR -- is to make a list, based on the RFC, not the code, of things that might need to be tested, and then compare the code against that list. I'm happy to go through and do it, or you can and post it somewhere. :)

(In case it wasn't clear, the code itself seems good to me! Nice work as usual.)

@eddyb eddyb force-pushed the impl-trait branch 3 times, most recently from c32f829 to c0b7f97 Compare August 10, 2016 21:04
@nikomatsakis
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Aug 11, 2016

📌 Commit c0b7f97 has been approved by nikomatsakis

@bors
Copy link
Contributor

bors commented Aug 12, 2016

☔ The latest upstream changes (presumably #35592) made this pull request unmergeable. Please resolve the merge conflicts.

@eddyb
Copy link
Member Author

eddyb commented Aug 12, 2016

@bors r=nikomatsakis

@bors
Copy link
Contributor

bors commented Aug 12, 2016

📌 Commit 23f0494 has been approved by nikomatsakis

@bors
Copy link
Contributor

bors commented Aug 12, 2016

⌛ Testing commit 23f0494 with merge f55ac69...

bors added a commit that referenced this pull request Aug 12, 2016
Implement `impl Trait` in return type position by anonymization.

This is the first step towards implementing `impl Trait` (cc #34511).
`impl Trait` types are only allowed in function and inherent method return types, and capture all named lifetime and type parameters, being invariant over them.
No lifetimes that are not explicitly named lifetime parameters are allowed to escape from the function body.
The exposed traits are only those listed explicitly, i.e. `Foo` and `Clone` in `impl Foo + Clone`, with the exception of "auto traits" (like `Send` or `Sync`) which "leak" the actual contents.

The implementation strategy is anonymization, i.e.:
```rust
fn foo<T>(xs: Vec<T>) -> impl Iterator<Item=impl FnOnce() -> T> {
    xs.into_iter().map(|x| || x)
}

// is represented as:
type A</*invariant over*/ T> where A<T>: Iterator<Item=B<T>>;
type B</*invariant over*/ T> where B<T>: FnOnce() -> T;
fn foo<T>(xs: Vec<T>) -> A<T> {
    xs.into_iter().map(|x| || x): $0 where $0: Iterator<Item=$1>, $1: FnOnce() -> T
}
```
`$0` and `$1` are resolved (to `iter::Map<vec::Iter<T>, closure>` and the closure, respectively) and assigned to `A` and `B`, after checking the body of `foo`. `A` and `B` are *never* resolved for user-facing type equality (typeck), but always for the low-level representation and specialization (trans).

The "auto traits" exception is implemented by collecting bounds like `impl Trait: Send` that have failed for the obscure `impl Trait` type (i.e. `A` or `B` above), pretending they succeeded within the function and trying them again after type-checking the whole crate, by replacing `impl Trait` with the real type.

While passing around values which have explicit lifetime parameters (of the function with `-> impl Trait`) in their type *should* work, regionck appears to assign inference variables in *way* too many cases, and never properly resolving them to either explicit lifetime parameters, or `'static`.
We might not be able to handle lifetime parameters in `impl Trait` without changes to lifetime inference, but type parameters can have arbitrary lifetimes in them from the caller, so most type-generic usecases (or not generic at all) should not run into this problem.

cc @rust-lang/lang
@bors
Copy link
Contributor

bors commented Aug 12, 2016

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-cargotest, auto-linux-64-cross-freebsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-no-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-no-mir, auto-win-msvc-64-opt-rustbuild
Approved by: nikomatsakis
Pushing f55ac69 to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants