Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HRTBs: "implementation is not general enough", but it is #70263

Open
comex opened this issue Mar 22, 2020 · 32 comments
Open

HRTBs: "implementation is not general enough", but it is #70263

comex opened this issue Mar 22, 2020 · 32 comments
Labels
A-closures Area: Closures (`|…| { … }`) A-higher-ranked Area: Higher-ranked things (e.g., lifetimes, types, trait bounds aka HRTBs) A-lifetimes Area: Lifetimes / regions A-trait-system Area: Trait system C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-lang Relevant to the language team, which will review and decide on the PR/issue.

Comments

@comex
Copy link
Contributor

comex commented Mar 22, 2020

Playground link

trait MyFn<'a> {}
impl<'a, F> MyFn<'a> for F where
    F: FnOnce(&'a i32) -> &'a i32 {}
    
fn foo<F>(f: F) where F: for<'a> MyFn<'a> {}
// This works:
// fn foo<F>(f: F) where F: for<'a> FnOnce(&'a i32) -> &'a i32 {}

fn main() {
    foo(|x: &i32| -> &i32 { x });
}

produces:

error: implementation of `MyFn` is not general enough
  --> src/main.rs:10:5
   |
1  | trait MyFn<'a> {}
   | ----------------- trait `MyFn` defined here
...
10 |     foo(|x: &i32| -> &i32 { x });
   |     ^^^ implementation of `MyFn` is not general enough
   |
   = note: `MyFn<'1>` would have to be implemented for the type `[closure@src/main.rs:10:9: 10:32]`, for any lifetime `'1`...
   = note: ...but `MyFn<'_>` is actually implemented for the type `[closure@src/main.rs:10:9: 10:32]`, for some specific lifetime `'2`

But clearly it actually is implemented for any lifetime.

@jonas-schievink jonas-schievink added A-lifetimes Area: Lifetimes / regions A-trait-system Area: Trait system C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-lang Relevant to the language team, which will review and decide on the PR/issue. labels Mar 22, 2020
@comex
Copy link
Contributor Author

comex commented Mar 22, 2020

Update: This appears to only affect closures. It works with fns:

fn bar(x: &i32) -> &i32 { x }
foo(bar); // OK

as well as with a type that manually implements FnOnce:

struct Manual;
impl<'a> FnOnce<(&'a i32,)> for Manual {
    type Output = &'a i32;
    extern "rust-call" fn call_once(self, (x,): (&'a i32,)) -> &'a i32 { x }
}
foo(Manual); // OK

@comex
Copy link
Contributor Author

comex commented Mar 23, 2020

In fact, we can make closures work by adding an extra bound:

trait MyFn<'a> {}
impl<'a, F> MyFn<'a> for F where
    F: FnOnce(&'a i32) -> &'a i32 {}

fn foo<F>(f: F) where
    F: for<'a> FnOnce(&'a i32) -> &'a i32, // <-- extra bound
    F: for<'a> MyFn<'a> {}

fn main() {
    foo(|x: &i32| -> &i32 { x }); // OK
}

On the other hand, if we instead add F: FnOnce(&'static i32) -> &'static i32, the error is still there. So the extra bound is presumably affecting how the type of the closure is inferred.

And if I add both extra bounds... I get a type error in the function definition!? Just filed that as #70290.

@Ekleog
Copy link

Ekleog commented May 3, 2020

I've recently hit this issue with futures, where as far as I understand the proposed workaround cannot work because there is no way to write the extra bound.

The “idea” I'm trying to do is https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=71374a2f88a2650391dba9903c733ed5 (where I've replaced Future with Debug for ease of example): take as parameter a closure that returns a future capturing its arguments.

The best I could do to implement that without HKTs is https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=bef14e19fdf52d4df695f12ca493f8a1, which looks like it should work (like in this issue) ; but stops compiling as soon as a closure is used instead of a raw function.

@comex
Copy link
Contributor Author

comex commented May 3, 2020

There's a way to make the workaround work by supplying the bound on the caller side. Based on your example:

// A function that just returns its argument, but serves as a type hint:
fn dummy<F>(f: F) -> F where F: for<'a> Fn(&'a u8) -> &'a u8 {
    f
}

// Doesn't work: foo(|x| x);
// Does work:
foo(dummy(|x| x));

This can be made a little less verbose by using a macro (and lifetime elision for good measure):

macro_rules! typed_closure {
    (($($bound:tt)*), $closure:expr) => { {
        fn _typed_closure_id<F>(f: F) -> F where F: $($bound)* { f }
        _typed_closure_id($closure)
    } }
}

foo(typed_closure!((Fn(&u8) -> &u8), |x| x));

Still, it is annoying that this is required.

@oberien
Copy link
Contributor

oberien commented Sep 9, 2020

This error doesn't only appear with closures, but also with function pointer types and Debug-derive. The following code results in a similar HRTB-based error (playground):

#[derive(Debug)]
enum Foo {
    Bar(fn(&())),
}

Error:

error: implementation of `std::fmt::Debug` is not general enough
   --> src/lib.rs:3:9
    |
3   |       Bar(fn(&())),
    |           ^^^^^^^ implementation of `std::fmt::Debug` is not general enough
    |
    = note: `std::fmt::Debug` would have to be implemented for the type `for<'r> fn(&'r ())`
    = note: ...but `std::fmt::Debug` is actually implemented for the type `fn(&'0 ())`, for some specific lifetime `'0`
    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

@fleabitdev
Copy link

Possibly related, with a confusing error message:

trait MyTrait { }
impl<'a> MyTrait for &'a i32 { }

fn example<R: MyTrait, F: Fn(&i32) -> R>(_f: F) { }

fn main() {
    let func: fn(&i32) -> &i32 = |x| x;
    example(func);
}
error: implementation of `FnOnce` is not general enough
   --> src/main.rs:8:5
    |
8   |       example(func);
    |       ^^^^^^^ implementation of `FnOnce` is not general enough
    |
    = note: `FnOnce<(&'0 i32,)>` would have to be implemented for the type `for<'r> fn(&'r i32) -> &'r i32`, for some specific lifetime `'0`...
    = note: ...but `FnOnce<(&i32,)>` is actually implemented for the type `for<'r> fn(&'r i32) -> &'r i32`

This made it difficult to write a generic function which boxes and wraps a user-provided closure. I encountered the error whenever an HRTB constrained the closure's return value to have a lifetime matching one of its parameters. (The error still occurs if the R: MyTrait bound is removed.)

Adding a free lifetime parameter to the example function, and to MyTrait, suppresses the error. However, this isn't always possible; a free lifetime must accept all lifetimes, including 'static, so it will prevent F from being called with a reference to data constructed on the stack.

I did find a workaround, but it requires nightly. It exploits the fact that the type checker seems to be better at normalizing associated types in trait where-clauses rather than function where-clauses:

#![feature(unboxed_closures)]

trait MyTrait { }
impl<'a> MyTrait for &'a i32 { }

trait MyTraitOutput<Args>: Fn<Args> { }
impl<Args, F> MyTraitOutput<Args> for F
where
    F: Fn<Args>,
    <F as FnOnce<Args>>::Output: MyTrait
{ }

fn example<F: for<'a> Fn<(&'a i32,)>>(_f: F)
where
    F: for<'a> MyTraitOutput<(&'a i32,)>
{
    //call the closure and store its result. then, pass the result to methods on 
    //MyTraitOutput which forward to methods on MyTrait
}

@angelsl
Copy link
Contributor

angelsl commented Jan 20, 2021

I think I'm running into this issue too, but I'm not too sure:

use std::future::Future;

struct A {}
struct B {}

async fn f<'a, G, Fut>(arg: &'a A, g: G)
where
    for<'b> G: Fn(&'a A, &'b B) -> Fut,
    Fut: Future<Output = String> + 'a,
{
    let b = B {};
    let ans = g(arg, &b).await;
}

fn main() {
    f(&A {}, async_func);
}

async fn async_func(a: &A, b: &B) -> String {
    "test".to_owned()
}

This produces:

error: implementation of `FnOnce` is not general enough
   --> src/main.rs:16:5
    |
16  |       f(&A {}, async_func);
    |       ^ implementation of `FnOnce` is not general enough
    |
    = note: `FnOnce<(&A, &'0 B)>` would have to be implemented for the type `for<'_, '_> fn(&A, &B) -> impl Future {async_func}`, for some specific lifetime `'0`...
    = note: ...but `FnOnce<(&A, &'b B)>` is actually implemented for the type `for<'_, '_> fn(&A, &B) -> impl Future {async_func}`

error: aborting due to previous error

error: could not compile `playground`

@mikeyhew
Copy link
Contributor

@comex I looked at the logs when compiling your two versions and noticed this log message is different in each case:

case 1 (not working), where foo has this signature:

fn foo<F>(f: F) where F: for<'a> MyFn<'a> {}
DEBUG rustc_typeck::check::closure check_closure(
    opt_kind=None,
    expected_sig=None
)

case 2 (working), where foo has this signature:

fn foo<F>(f: F) where F: for<'a> FnOnce(&'a i32) -> &'a i32 {}
DEBUG rustc_typeck::check::closure check_closure(
    opt_kind=Some(FnOnce),
    expected_sig=Some(ExpectedSig {
        cause_span: Some(src/lib.rs:10:5: 10:8 (#0)),
        sig: Binder(
            ([&'a i32]; c_variadic: false)->&'a i32,
            [Region(BrNamed(DefId(0:10 ~ foo[df6b]::foo::'a), 'a))]
        )
    })
)

(I made these logs looks prettier by hand, I wonder if there's a way to automate that)

for reference, this is the call expression that's being checked, with the closure inside it

foo(|x: &i32| -> &i32 { x })

What this is showing is that:

  • in the working version, where foo has the F: for<'a> FnOnce bound,
    • when the call expression is checked, it calls check_closure on the closure with an expected signature
  • whereas in the not-working version, where foo just has the F: for<'a> MyFn bound,
    • when the call expression is checked, it calls check_closure on the closure without an expected signature

I'm going out on a limb here, but it seems to me like this is how it works:

  • there's special treatment of call expressions when the callee (the function be called, foo in this case) has an FnOnce bound on the type of the argument, which causes there to be an expected signature for the closure when it's typechecked, and then
  • when a closure is typechecked with an expected signature with a binder like this one, it infers the HRTB signature; and without an expected signature to help it out, it doesn't

@SkiFire13
Copy link
Contributor

when a closure is typechecked with an expected signature with a binder like this one, it infers the HRTB signature; and without an expected signature to help it out, it doesn't

To add more informations on this, the code responsible for creating the closure signature when there's an fn-like bound is this:

let bound_sig = expected_sig.sig.map_bound(|sig| {
self.tcx.mk_fn_sig(
sig.inputs().iter().cloned(),
sig.output(),
sig.c_variadic,
hir::Unsafety::Normal,
Abi::RustCall,
)
});

it reuses the signature of the provided bound, which is a HRTB, so everything is how we expect it to be. In fact, debugging the inferred signature, I get something like for<'r> extern "rust-call" fn(&'r i32) -> &'r i32. However the code responsible for creating the closure signature when that type of bound is missing is this:

let result = ty::Binder::bind_with_vars(
self.tcx.mk_fn_sig(
supplied_arguments,
supplied_return,
decl.c_variadic,
hir::Unsafety::Normal,
Abi::RustCall,
),
bound_vars,
);

It creates a new function signature from the provided (or maybe inferred) types, resulting in a signature like for<'r> extern "rust-call" fn(&'r i32) -> &i32. Surprisingly this is still a HRTB, but only the input types are late bound.

@Skepfyr
Copy link
Contributor

Skepfyr commented May 28, 2021

I just hit something that looks very similar but doesn't involve any closures at all. The most minimal I could get it is this:

fn bind_service<S>()
where
    S: for<'a> Service<&'a ()>,
{
    serve::<_, Wrapper<S>>();
}

fn serve<F, S: for<'a> Service<&'a (), Future = F>>() {}

trait Service<Request> {
    type Future;
}

struct Wrapper<S>(S);

impl<'a, S> Service<&'a ()> for Wrapper<S>
where
    S: for<'b> Service<&'b ()>,
{
    type Future = <S as Service<&'a ()>>::Future;
}

It compiles under chalk. Some changes that makes it compile are:

fn bind_service<S, F>()
where
    S: for<'a> Service<&'a (), Future = F>,

and

fn serve<S: for<'a> Service<&'a ()>>() {}

@QuineDot
Copy link

In the case of closures, there is more discussion on what I believe is the same phenomenon in Issue #58052.

@guswynn
Copy link
Contributor

guswynn commented Jun 28, 2021

Is this: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=343c55dbef8f6854799af4b8b328b6c1 another example of this?

error: implementation of `Helper` is not general enough
  --> src/main.rs:36:5
   |
36 |     par_iter_with_setup(v, setup, |iter| ())
   |     ^^^^^^^^^^^^^^^^^^^ implementation of `Helper` is not general enough
   |
   = note: `for<'a> fn(&'a Vec<String>) -> Vec<&'a str> {setup}` must implement `Helper<'0, Vec<String>>`, for any lifetime `'0`...
   = note: ...but it actually implements `Helper<'1, Vec<String>>`, for some specific lifetime `'1`

To me this is saying, you have a fn that works for any lifetime, but its saying that is implements the trait for only 1 lifetime

I am following advice in https://users.rust-lang.org/t/hrtb-on-multiple-generics/34255 to pull my for<'a> bound into a trait, but I'm not sure why https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=685c5d84ee54f6de47fd2b5dc3412c22 works but my example doesnt

@drewcrawford
Copy link
Contributor

I've hit a similar issue involving futures, but I can't seem to adapt any of the workarounds to work on closures that case:

//idea here is AsyncClosure argument and return future have the same lifetime
trait AsyncClosure<'a,Argument,Output> {
        type Fut: std::future::Future<Output = Output> + 'a;
        fn call(self, arg: &'a Argument) -> Self::Fut;
}

//blanket impl
impl<'a, Fu: 'a, F,Argument: 'static,Output> AsyncClosure<'a,Argument,Output> for F
where
    F: FnOnce(&'a Argument) -> Fu,
    Fu: std::future::Future<Output = Output> + 'a,
{
    type Fut = Fu;
    fn call(self, rt: &'a Argument) -> Fu {
        self(rt)
    }
}

async fn with_async_closure< C,R>(c: C) -> R where for<'a> C: AsyncClosure<'a,u8,R> {
	let a = 3; //closure borrows this
	c.call(&a) //returned future borrows it as well
	.await
	//no longer borrowed here
}

async fn function_target(arg:&u8) {
    println!("{:?}",arg);
}

async fn amain() {
    with_async_closure(function_target); //works on bare fn
   with_async_closure( |arg| async { function_target(arg) } ); //error: implementation of `AsyncClosure` is not general enough
}

@eira-fransham
Copy link

eira-fransham commented Aug 26, 2021

I've hit a similar issue in parking_lot. We want to generalise MappedMutexGuard to be able to have owned data instead of just &mut (PR here Amanieu/parking_lot#290), but we cannot give the mapping closure a type with the full lifetime of the mutex, as it would allow UB:

impl<'a, R: RawMutex + 'a, T: ?Sized + 'a> MutexGuard<'a, R, T> {
    // ..

    pub fn map<U, F>(s: Self, f: F) -> MappedMutexGuard<'a, R, U>
    where
        F: FnOnce(&'a mut T) -> U,
    {
        // ..
    }

    // ..
}

#[test]
fn test_map() {
    use crate::{lock_api::RawMutex, Mutex, MutexGuard};

    const FOO: usize = 0;

    static OUTER: Mutex<&usize> = Mutex::const_new(RawMutex::INIT, &FOO);
    static M: Mutex<usize> = Mutex::const_new(RawMutex::INIT, 0);

    let guard = MutexGuard::map(M.lock(), |inner: &'static mut usize| {
        *OUTER.lock() = inner;
    });
    drop(guard);

    let outer: &usize = &*OUTER.lock();

    assert_eq!(*outer, 0);

    *M.lock() = 1;

    assert_eq!(*outer, 1);
}

So we want to require that the mapping closure is valid for any lifetime. This makes the code valid since the only way to get the inner data of the MutexGuard/MappedMutexGuard is via a &self/&mut self method, which means that the user can never get a reference to the data that lasts as long as the mutex. This then causes an issue - while you can annotate for<'any> F: FnOnce(&'any mut T) -> &'any mut U, there's no equivalent for a closure that takes/returns a generic type (which could be &'any mut U, Ref<'any, U>, etc). We can solve this using a "shim" trait:

pub trait FnOnceShim<'a, T: 'a> {
    type Output: 'a;

    fn call(self, input: T) -> Self::Output;
}

impl<'a, F, In, Out> FnOnceShim<'a, In> for F
where
    F: FnOnce(In) -> Out,
    In: 'a,
    Out: 'a,
{
    type Output = Out;

    fn call(self, input: In) -> Self::Output {
        self(input)
    }
}

impl<'a, R: RawMutex + 'a, T: ?Sized + 'a> MutexGuard<'a, R, T> {
    // ..

    pub fn map<F>(
        s: Self,
        f: F,
    ) -> MappedMutexGuard<'a, R, <F as FnOnceShim<'a, &'a mut T>>::Output>
    where
        for<'any> F: FnOnceShim<'any, &'any mut T>,
    {
        // ..
    }

    // ..
}

This works, and correctly disallows invalid code while allowing the same use-cases as the old code which was specific to &mut, plus also allowing mapping functions which return types like Ref<'_, T> or something like that. Unfortunately, actually using this API becomes an issue, as Rust refuses to infer a for<'any> bound automatically and you need to force it:

use parking_lot::{Mutex, MutexGuard};

let m = Mutex::new((0, 0));
let guard = MutexGuard::map(
    m.lock(),
    |inner: &mut (usize, usize)| -> &mut usize { &mut inner.0 },
);

Causes this error:

error: implementation of `FnOnceShim` is not general enough
 --> src/mutex.rs:118:13
  |
7 | let guard = MutexGuard::map(
  |             ^^^^^^^^^^^^^^^ implementation of `FnOnceShim` is not general enough
  |
  = note: `[closure@src/mutex.rs:9:5: 9:38]` must implement `FnOnceShim<'0, &'0 mut (i32, i32)>`, for any lifetime `'0`...
  = note: ...but it actually implements `FnOnceShim<'_, &'any mut (i32, i32)>`

The workaround is to make an identity function with a more-specific HRTB, which allows you to manually specify that the return lifetime is generic.

fn annotate<T, U, F>(f: F) -> F
where
    F: FnOnce(&mut T) -> &mut U,
{
    f
}

let m = Mutex::new((0, 0));

let mapper = annotate(|inner: &mut (_, _)| &mut inner.0);

let guard = MutexGuard::map(m.lock(), mapper);

// If you need to use some other type with a lifetime param, you need to write a new `annotate` fn
struct MutWrapper<'a, T>(&'a mut T);

fn annotate_wrapper<T, U, F>(f: F) -> F
where
    F: FnOnce(&mut T) -> MutWrapper<'_, U>
{
    f
}

@danielhenrymantilla
Copy link
Contributor

Workaround

Btw, for those stumbling upon this issue, and until rust-lang/rfcs#3216 gets accepted/implemented, there is https://docs.rs/higher-order-closure as a workaround.

@Wandalen
Copy link

Wandalen commented Apr 23, 2022

Hi! Problem I encountered looks somehow related to the bug. Maybe you can suggest a workaround?

That works. But compiler throws exception "implementation of ReactorInterface is not general enough" if ReactorInterface has a mutable associated function. Please have a look. Why is that so? Is it really related to the bug? Does it has a solution?

@SkiFire13
Copy link
Contributor

@Wandalen I think that error is right. The problem in your code is that you're relying on covariance to convert an HashMap<usize, Box<HandlerOfEventDyn>> to a HashMap<usize, Box<HandlerOfEventDynWithLifetime<'a>>>, but the mutable version returns a mutable reference, which is invariant.

If this was allowed you could put a Box<HandlerOfEventDynWithLifetime<'a>> inside the map, even though it is valid only for a specific lifetime 'a instead of every lifetime, like self.handlers's elements are expected to.

@Wandalen
Copy link

Wandalen commented Apr 23, 2022

Thanks @SkiFire13. If so what is proper solution of the problem? I am aware about variance, not sure it should work like you explain, because Box<HandlerOfEventDyn> and Box<HandlerOfEventDynWithLifetime<'a> is the same type. My understanding is so that compiler should spell 'a as any lifetime in the context. Removing aliases of types make it's more obvious that type is the same:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c723c662f168e306cef432f269180128

If my understanding is wrong what is solution of the problem? It seems like Rust does allow use generic programming only for trivial cases, but it's pain if I try to implement any system of types which is more complex than that.

@danielhenrymantilla
Copy link
Contributor

It is not the same type. Removing the type alias, but imagining some trait alias Trait<'lt> to reduce it to one generic lifetime parameter, we have:

  • dyn for<'any> Trait<'any> for your HandleOfEventDyn,
  • dyn Trait<'a>, for some specific 'a, for your HandlerOfEventDynWithLifetime<'a>

The former represents an erased type that satisifies the Trait contract for each and every lifetime.
The latter represents an erased type that satisfies the Trait for that specific lifetime 'a.

Obviously the former subtypes the latter, but they're still distinct types. And subtyping does not apply behind &mut, since otherwise the user of that handlers_mut() method could try to overwrite the value type with its own erased type, with only the constraint that it satisfy the Trait for that specific lifetime (instead of for all lifetimes!). And then you'd be having a value not necessarily satisfying-Trait -for-each-and-every-lifetime inside your Reactor, contradicting what the type system states.

@Wandalen
Copy link

Wandalen commented Apr 23, 2022

I see. dyn for<'any> could be used only on places where trait should be, In a struct as a field dyn Trait<'a> could only be used.

Attempts to eliminate either of the types fail. What is a proper solution?

I am thinking about using pointer instead of reference here:

pub struct Event1<'a> {
    a: &'a u32,
}

But even if it will remove the blocker, I don't feel it's a nice workaround.

Other thoughts I have using dyn more extensively. Although not sure how. I made such an attempt, but encountered the same problem on another level.

@Wandalen
Copy link

Wandalen commented Apr 24, 2022

Is that a proper of my solution to use pointers instead of references?

pub struct Event1 {
    a: *mut u32,
}

impl Event1 {
    pub fn a<'a>(&'a self) -> &'a u32 {
        unsafe { std::mem::transmute::<_, &'a _>(self.a) }
    }

    pub fn a_mut<'a>(&'a mut self) -> &'a mut u32 {
        unsafe { std::mem::transmute::<_, &'a mut _>(self.a) }
    }
}

But what to do if field a has a ref?..

struct Struct1<'a> {
    b: &'a i32,
}

pub struct Event {
    a: *mut Struct1<x>,
}

@SkiFire13
Copy link
Contributor

@Wandalen it depends on how you create Event1 and what should its raw pointer fields represent.

Anyway since your question isn't related to this issue I suggest you to open a thread on https://users.rust-lang.org/ or ask on the community discord server.

@Wandalen
Copy link

Wandalen commented Apr 24, 2022

Sure. Does not it? Thanks.

@jonhoo
Copy link
Contributor

jonhoo commented Jun 9, 2022

Another example where closure type inference works only if the closure is not assigned to a variable:

fn x<F>(_: F)
where
    for<'c> F: Fn(&'c ()) -> &'c (),
{
}

fn main() {
    // This works
    x(|c| c);

    // This fails with "implementation of `FnOnce` is not general enough"
    let z = |c| c;
    x(z);
    
    // This fails with "expected reference &() found reference &'c ()"
    let z = |c: &_| c;
    x(z);
}

@nam178
Copy link

nam178 commented Nov 13, 2022

I believe I am hitting a similar problem, I'm trying to create a function that takes a string, or a closure that returns a string:
Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=2cba77d590430996665c8f174c27b6ed

// Represents anything that can generate a string
pub(crate) trait ErrorMessageProvider<TTarget>
{
    fn provide(&self, target: &TTarget) -> String;
}

// Closure that returns a string
impl<TTarget, TFunction: Fn(&TTarget) -> String>
    ErrorMessageProvider<TTarget> for TFunction
{
    fn provide(&self, target: &TTarget) -> String {
        self(target)
    }
}

// A string
impl<TTarget> ErrorMessageProvider<TTarget> for &str
{
    fn provide(&self, _target: &TTarget) -> String {
        (*self).to_owned()
    }
}

// Function that takes a String, or a closure that returns a string
fn take<TTarget>(provider: impl ErrorMessageProvider<TTarget>, target: TTarget) {
    
}

fn main() {
    take("BAR", 100); // Works
    take(|x| format!("{:?}", x), 100); // Error: implementation of `FnOnce` is not general enough
}

@frederikhors
Copy link

@davidzeng0
Copy link
Contributor

for<'a> can be read as "for all choices of 'a", and basically produces an infinite list of trait bounds that F must satisfy.

I believe the wording in the documentation is mistaken, and the compiler is also mistaken for requiring a generic implementation of 'a for any 'a. From my understanding, the necessity of the HRTB concept is as follows, and if I am correct, then HRTBs are implemented incorrectly.

In rust, the generic lifetime <'0> has the minimum bound that the external value with lifetime '0 is valid inside the generic function or trait. Setting a bound of '0: '1 ensures that the external &'0 T :> &'1 U.

Because external generic lifetimes must always be valid internally, it is impossible to specify an arbitrary internal generic lifetime 'a that is usable/referenceable externally for passing, for example, FnOnce's with generic lifetime parameters.

Such a lifetime 'a is one that has no external minimum bound, but has a maximum internal bound of 'local_vars, meaning that strictly 'a :< 'local_vars without setting any minimum bounds on 'local_vars, and it is also automatically true that the generics '0 and '1 must be '0: 'a and '1: 'a.

If 'a is 'static, then '0: 'static and '1: 'static. Except, 'a cannot be 'static because 'a :< 'local_vars would no longer be true.

Let's introduce the hypothetical keyword internal to "export" our internal lifetimes so that they may be referenced by generic FnOnce's, allowing us to write the following code

fn take_closure<'b, internal 'a, F>(func: F, x: &'b i32) where F: FnOnce(&'a i32) -> &'a i32 {
    let z = &20;
    let val = func(x);
    
    println!("{}", val);

    let val = func(z);
    
    println!("{}", val);
}

// Ok, this works
take_closure(|x| x, &3);

This desugars to the following

'b: {
    let x: &'b i32;

    fn take_closure<F>(func: F, x: &'b i32) {
        'local_vars: {
            let z: &'local_vars i32 = &20;

            'a where F: FnOnce(&'a i32) -> &'a i32, 'local_vars: 'a {
                let val = func(x);
    
                println!("{}", val);
                
                let val = func(z);

                println!("{}", val);
            }
        }
    }
}

'c: {
    let closure = |x: &'c i32| -> &'c i32 { x };
    let x: &'static i32 = &3;
    
    // Ok, this works
    take_closure<'static, 'c, _>(closure, x);
}

which is completely OK.

Ideally, we would like to be able to write something like this, in real rust

// error: returning this value requires that `'c` must outlive `'static`
pub fn works<'b, 'c>(x: &'b i32, y: &'c i32) {
    take_closure(|x| { x.min(y) }, x);
}

however, the following works

pub fn ok<'b, 'c>(x: &'b i32, y: &'c i32) {
    struct A<'a, 'c> where 'c: 'a {
        y: &'c i32,
        phantom: std::marker::PhantomData<&'a ()>
    }
    
    impl<'a, 'c> A<'a, 'c> {
        fn run(&'_ self, x: &'a i32) -> &'a i32 {
            x.min(self.y)
        }
    }
    
    let val = A { y, phantom: std::marker::PhantomData }.run(x);
    
    println!("{}", val);
}

and also works

pub fn ok2<'b, 'c>(x: &'b i32, y: &'c i32) {
    trait A<'a>: FnOnce(&'a i32) -> &'a i32 {}
    impl<'a, T> A<'a> for T where T: FnOnce(&'a i32) -> &'a i32 {} 
    
    fn make_a<'a, 'c>(y: &'c i32) -> impl A<'a> where 'c: 'a {
        |x: &'a i32| -> &'a i32 {
            x.min(y)
        }
    }
    
    // This fails because FnOnce is not general enough,
    // however it runs perfectly fine below
    // ```
    // take_closure(make_a(y), x);
    // ```
    
    let val = make_a(y)(x);
    
    println!("{}", val);
}

See playground link.

The issue is that the docs state that for<'a> means for all choices of 'a, whether that be 'static, a bounded lifetime, etc...
And the compiler ensures this by making sure that an impl<'a> Trait<'a> where UNBOUNDED exists for any 'a.
But can that mean 'a = 'static?

Given the following function,

fn take_closure<'b, F>(func: F, x: &'b i32) where F: for<'a> FnOnce(&'a i32) -> &'a i32;

how would 'b know that it needs to be 'b: 'static if 'a: 'static?
It also would not make sense to impl Trait<'static>, as then the compiler would complain that it's not generic enough.

Therefore, for<'a> should not be worded as "for all choices of 'a", but rather "for any arbitrary 'a that has no minimum bounds by any 'lifetime where 'lifetime: 'local_vars".

This would enable us to write the following, in which 'a is an arbitrary unbounded lifetime

fn make_closure<'a>() -> impl (FnOnce(&'a i32) -> &'a i32) + 'static {
    |x| { x }
}

@fmease fmease added A-closures Area: Closures (`|…| { … }`) A-higher-ranked Area: Higher-ranked things (e.g., lifetimes, types, trait bounds aka HRTBs) labels Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-closures Area: Closures (`|…| { … }`) A-higher-ranked Area: Higher-ranked things (e.g., lifetimes, types, trait bounds aka HRTBs) A-lifetimes Area: Lifetimes / regions A-trait-system Area: Trait system C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests