-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HRTBs: "implementation is not general enough", but it is #70263
Comments
Update: This appears to only affect closures. It works with fn bar(x: &i32) -> &i32 { x }
foo(bar); // OK as well as with a type that manually implements struct Manual;
impl<'a> FnOnce<(&'a i32,)> for Manual {
type Output = &'a i32;
extern "rust-call" fn call_once(self, (x,): (&'a i32,)) -> &'a i32 { x }
}
foo(Manual); // OK |
In fact, we can make closures work by adding an extra bound: trait MyFn<'a> {}
impl<'a, F> MyFn<'a> for F where
F: FnOnce(&'a i32) -> &'a i32 {}
fn foo<F>(f: F) where
F: for<'a> FnOnce(&'a i32) -> &'a i32, // <-- extra bound
F: for<'a> MyFn<'a> {}
fn main() {
foo(|x: &i32| -> &i32 { x }); // OK
} On the other hand, if we instead add And if I add both extra bounds... I get a type error in the function definition!? Just filed that as #70290. |
I've recently hit this issue with futures, where as far as I understand the proposed workaround cannot work because there is no way to write the extra bound. The “idea” I'm trying to do is https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=71374a2f88a2650391dba9903c733ed5 (where I've replaced The best I could do to implement that without HKTs is https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=bef14e19fdf52d4df695f12ca493f8a1, which looks like it should work (like in this issue) ; but stops compiling as soon as a closure is used instead of a raw function. |
There's a way to make the workaround work by supplying the bound on the caller side. Based on your example: // A function that just returns its argument, but serves as a type hint:
fn dummy<F>(f: F) -> F where F: for<'a> Fn(&'a u8) -> &'a u8 {
f
}
// Doesn't work: foo(|x| x);
// Does work:
foo(dummy(|x| x)); This can be made a little less verbose by using a macro (and lifetime elision for good measure): macro_rules! typed_closure {
(($($bound:tt)*), $closure:expr) => { {
fn _typed_closure_id<F>(f: F) -> F where F: $($bound)* { f }
_typed_closure_id($closure)
} }
}
foo(typed_closure!((Fn(&u8) -> &u8), |x| x)); Still, it is annoying that this is required. |
This error doesn't only appear with closures, but also with function pointer types and #[derive(Debug)]
enum Foo {
Bar(fn(&())),
} Error: error: implementation of `std::fmt::Debug` is not general enough
--> src/lib.rs:3:9
|
3 | Bar(fn(&())),
| ^^^^^^^ implementation of `std::fmt::Debug` is not general enough
|
= note: `std::fmt::Debug` would have to be implemented for the type `for<'r> fn(&'r ())`
= note: ...but `std::fmt::Debug` is actually implemented for the type `fn(&'0 ())`, for some specific lifetime `'0`
= note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info) |
Possibly related, with a confusing error message: trait MyTrait { }
impl<'a> MyTrait for &'a i32 { }
fn example<R: MyTrait, F: Fn(&i32) -> R>(_f: F) { }
fn main() {
let func: fn(&i32) -> &i32 = |x| x;
example(func);
}
This made it difficult to write a generic function which boxes and wraps a user-provided closure. I encountered the error whenever an HRTB constrained the closure's return value to have a lifetime matching one of its parameters. (The error still occurs if the Adding a free lifetime parameter to the I did find a workaround, but it requires nightly. It exploits the fact that the type checker seems to be better at normalizing associated types in trait where-clauses rather than function where-clauses: #![feature(unboxed_closures)]
trait MyTrait { }
impl<'a> MyTrait for &'a i32 { }
trait MyTraitOutput<Args>: Fn<Args> { }
impl<Args, F> MyTraitOutput<Args> for F
where
F: Fn<Args>,
<F as FnOnce<Args>>::Output: MyTrait
{ }
fn example<F: for<'a> Fn<(&'a i32,)>>(_f: F)
where
F: for<'a> MyTraitOutput<(&'a i32,)>
{
//call the closure and store its result. then, pass the result to methods on
//MyTraitOutput which forward to methods on MyTrait
} |
I think I'm running into this issue too, but I'm not too sure: use std::future::Future;
struct A {}
struct B {}
async fn f<'a, G, Fut>(arg: &'a A, g: G)
where
for<'b> G: Fn(&'a A, &'b B) -> Fut,
Fut: Future<Output = String> + 'a,
{
let b = B {};
let ans = g(arg, &b).await;
}
fn main() {
f(&A {}, async_func);
}
async fn async_func(a: &A, b: &B) -> String {
"test".to_owned()
} This produces:
|
Function pointers don't implement Debug if some of their arguments are references, as seen in rust-lang/rust#70263.
@comex I looked at the logs when compiling your two versions and noticed this log message is different in each case: case 1 (not working), where fn foo<F>(f: F) where F: for<'a> MyFn<'a> {}
case 2 (working), where
(I made these logs looks prettier by hand, I wonder if there's a way to automate that) for reference, this is the call expression that's being checked, with the closure inside it foo(|x: &i32| -> &i32 { x }) What this is showing is that:
I'm going out on a limb here, but it seems to me like this is how it works:
|
To add more informations on this, the code responsible for creating the closure signature when there's an fn-like bound is this: rust/compiler/rustc_typeck/src/check/closure.rs Lines 395 to 403 in 3f46b82
it reuses the signature of the provided bound, which is a HRTB, so everything is how we expect it to be. In fact, debugging the inferred signature, I get something like rust/compiler/rustc_typeck/src/check/closure.rs Lines 587 to 596 in 3f46b82
It creates a new function signature from the provided (or maybe inferred) types, resulting in a signature like |
I just hit something that looks very similar but doesn't involve any closures at all. The most minimal I could get it is this: fn bind_service<S>()
where
S: for<'a> Service<&'a ()>,
{
serve::<_, Wrapper<S>>();
}
fn serve<F, S: for<'a> Service<&'a (), Future = F>>() {}
trait Service<Request> {
type Future;
}
struct Wrapper<S>(S);
impl<'a, S> Service<&'a ()> for Wrapper<S>
where
S: for<'b> Service<&'b ()>,
{
type Future = <S as Service<&'a ()>>::Future;
} It compiles under chalk. Some changes that makes it compile are: fn bind_service<S, F>()
where
S: for<'a> Service<&'a (), Future = F>, and fn serve<S: for<'a> Service<&'a ()>>() {} |
In the case of closures, there is more discussion on what I believe is the same phenomenon in Issue #58052. |
Is this: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=343c55dbef8f6854799af4b8b328b6c1 another example of this?
To me this is saying, you have a fn that works for any lifetime, but its saying that is implements the trait for only 1 lifetime I am following advice in https://users.rust-lang.org/t/hrtb-on-multiple-generics/34255 to pull my |
I've hit a similar issue involving futures, but I can't seem to adapt any of the workarounds to work on closures that case: //idea here is AsyncClosure argument and return future have the same lifetime
trait AsyncClosure<'a,Argument,Output> {
type Fut: std::future::Future<Output = Output> + 'a;
fn call(self, arg: &'a Argument) -> Self::Fut;
}
//blanket impl
impl<'a, Fu: 'a, F,Argument: 'static,Output> AsyncClosure<'a,Argument,Output> for F
where
F: FnOnce(&'a Argument) -> Fu,
Fu: std::future::Future<Output = Output> + 'a,
{
type Fut = Fu;
fn call(self, rt: &'a Argument) -> Fu {
self(rt)
}
}
async fn with_async_closure< C,R>(c: C) -> R where for<'a> C: AsyncClosure<'a,u8,R> {
let a = 3; //closure borrows this
c.call(&a) //returned future borrows it as well
.await
//no longer borrowed here
}
async fn function_target(arg:&u8) {
println!("{:?}",arg);
}
async fn amain() {
with_async_closure(function_target); //works on bare fn
with_async_closure( |arg| async { function_target(arg) } ); //error: implementation of `AsyncClosure` is not general enough
} |
I've hit a similar issue in impl<'a, R: RawMutex + 'a, T: ?Sized + 'a> MutexGuard<'a, R, T> {
// ..
pub fn map<U, F>(s: Self, f: F) -> MappedMutexGuard<'a, R, U>
where
F: FnOnce(&'a mut T) -> U,
{
// ..
}
// ..
}
#[test]
fn test_map() {
use crate::{lock_api::RawMutex, Mutex, MutexGuard};
const FOO: usize = 0;
static OUTER: Mutex<&usize> = Mutex::const_new(RawMutex::INIT, &FOO);
static M: Mutex<usize> = Mutex::const_new(RawMutex::INIT, 0);
let guard = MutexGuard::map(M.lock(), |inner: &'static mut usize| {
*OUTER.lock() = inner;
});
drop(guard);
let outer: &usize = &*OUTER.lock();
assert_eq!(*outer, 0);
*M.lock() = 1;
assert_eq!(*outer, 1);
} So we want to require that the mapping closure is valid for any lifetime. This makes the code valid since the only way to get the inner data of the pub trait FnOnceShim<'a, T: 'a> {
type Output: 'a;
fn call(self, input: T) -> Self::Output;
}
impl<'a, F, In, Out> FnOnceShim<'a, In> for F
where
F: FnOnce(In) -> Out,
In: 'a,
Out: 'a,
{
type Output = Out;
fn call(self, input: In) -> Self::Output {
self(input)
}
}
impl<'a, R: RawMutex + 'a, T: ?Sized + 'a> MutexGuard<'a, R, T> {
// ..
pub fn map<F>(
s: Self,
f: F,
) -> MappedMutexGuard<'a, R, <F as FnOnceShim<'a, &'a mut T>>::Output>
where
for<'any> F: FnOnceShim<'any, &'any mut T>,
{
// ..
}
// ..
} This works, and correctly disallows invalid code while allowing the same use-cases as the old code which was specific to use parking_lot::{Mutex, MutexGuard};
let m = Mutex::new((0, 0));
let guard = MutexGuard::map(
m.lock(),
|inner: &mut (usize, usize)| -> &mut usize { &mut inner.0 },
); Causes this error:
The workaround is to make an identity function with a more-specific HRTB, which allows you to manually specify that the return lifetime is generic. fn annotate<T, U, F>(f: F) -> F
where
F: FnOnce(&mut T) -> &mut U,
{
f
}
let m = Mutex::new((0, 0));
let mapper = annotate(|inner: &mut (_, _)| &mut inner.0);
let guard = MutexGuard::map(m.lock(), mapper);
// If you need to use some other type with a lifetime param, you need to write a new `annotate` fn
struct MutWrapper<'a, T>(&'a mut T);
fn annotate_wrapper<T, U, F>(f: F) -> F
where
F: FnOnce(&mut T) -> MutWrapper<'_, U>
{
f
} |
WorkaroundBtw, for those stumbling upon this issue, and until rust-lang/rfcs#3216 gets accepted/implemented, there is https://docs.rs/higher-order-closure as a workaround. |
Hi! Problem I encountered looks somehow related to the bug. Maybe you can suggest a workaround? That works. But compiler throws exception "implementation of |
@Wandalen I think that error is right. The problem in your code is that you're relying on covariance to convert an If this was allowed you could put a |
Thanks @SkiFire13. If so what is proper solution of the problem? I am aware about variance, not sure it should work like you explain, because If my understanding is wrong what is solution of the problem? It seems like Rust does allow use generic programming only for trivial cases, but it's pain if I try to implement any system of types which is more complex than that. |
It is not the same type. Removing the type alias, but imagining some trait alias
The former represents an erased type that satisifies the Obviously the former subtypes the latter, but they're still distinct types. And subtyping does not apply behind |
I see. Attempts to eliminate either of the types fail. What is a proper solution? I am thinking about using pointer instead of reference here: pub struct Event1<'a> {
a: &'a u32,
} But even if it will remove the blocker, I don't feel it's a nice workaround. Other thoughts I have using |
Is that a proper of my solution to use pointers instead of references? pub struct Event1 {
a: *mut u32,
}
impl Event1 {
pub fn a<'a>(&'a self) -> &'a u32 {
unsafe { std::mem::transmute::<_, &'a _>(self.a) }
}
pub fn a_mut<'a>(&'a mut self) -> &'a mut u32 {
unsafe { std::mem::transmute::<_, &'a mut _>(self.a) }
}
} But what to do if field
|
@Wandalen it depends on how you create Anyway since your question isn't related to this issue I suggest you to open a thread on https://users.rust-lang.org/ or ask on the community discord server. |
Sure. Does not it? Thanks. |
Another example where closure type inference works only if the closure is not assigned to a variable: fn x<F>(_: F)
where
for<'c> F: Fn(&'c ()) -> &'c (),
{
}
fn main() {
// This works
x(|c| c);
// This fails with "implementation of `FnOnce` is not general enough"
let z = |c| c;
x(z);
// This fails with "expected reference &() found reference &'c ()"
let z = |c: &_| c;
x(z);
} |
I believe I am hitting a similar problem, I'm trying to create a function that takes a string, or a closure that returns a string:
|
I think I have a similar issue: https://users.rust-lang.org/t/this-line-make-rust-errors-and-i-cannot-understand-what-does-it-mean/87914/1. |
I believe the wording in the documentation is mistaken, and the compiler is also mistaken for requiring a generic implementation of In rust, the generic lifetime Because external generic lifetimes must always be valid internally, it is impossible to specify an arbitrary internal generic lifetime Such a lifetime If Let's introduce the hypothetical keyword fn take_closure<'b, internal 'a, F>(func: F, x: &'b i32) where F: FnOnce(&'a i32) -> &'a i32 {
let z = &20;
let val = func(x);
println!("{}", val);
let val = func(z);
println!("{}", val);
}
// Ok, this works
take_closure(|x| x, &3); This desugars to the following 'b: {
let x: &'b i32;
fn take_closure<F>(func: F, x: &'b i32) {
'local_vars: {
let z: &'local_vars i32 = &20;
'a where F: FnOnce(&'a i32) -> &'a i32, 'local_vars: 'a {
let val = func(x);
println!("{}", val);
let val = func(z);
println!("{}", val);
}
}
}
}
'c: {
let closure = |x: &'c i32| -> &'c i32 { x };
let x: &'static i32 = &3;
// Ok, this works
take_closure<'static, 'c, _>(closure, x);
} which is completely OK. Ideally, we would like to be able to write something like this, in real rust // error: returning this value requires that `'c` must outlive `'static`
pub fn works<'b, 'c>(x: &'b i32, y: &'c i32) {
take_closure(|x| { x.min(y) }, x);
} however, the following works pub fn ok<'b, 'c>(x: &'b i32, y: &'c i32) {
struct A<'a, 'c> where 'c: 'a {
y: &'c i32,
phantom: std::marker::PhantomData<&'a ()>
}
impl<'a, 'c> A<'a, 'c> {
fn run(&'_ self, x: &'a i32) -> &'a i32 {
x.min(self.y)
}
}
let val = A { y, phantom: std::marker::PhantomData }.run(x);
println!("{}", val);
} and also works pub fn ok2<'b, 'c>(x: &'b i32, y: &'c i32) {
trait A<'a>: FnOnce(&'a i32) -> &'a i32 {}
impl<'a, T> A<'a> for T where T: FnOnce(&'a i32) -> &'a i32 {}
fn make_a<'a, 'c>(y: &'c i32) -> impl A<'a> where 'c: 'a {
|x: &'a i32| -> &'a i32 {
x.min(y)
}
}
// This fails because FnOnce is not general enough,
// however it runs perfectly fine below
// ```
// take_closure(make_a(y), x);
// ```
let val = make_a(y)(x);
println!("{}", val);
} See playground link. The issue is that the docs state that Given the following function, fn take_closure<'b, F>(func: F, x: &'b i32) where F: for<'a> FnOnce(&'a i32) -> &'a i32; how would Therefore, This would enable us to write the following, in which fn make_closure<'a>() -> impl (FnOnce(&'a i32) -> &'a i32) + 'static {
|x| { x }
} |
Playground link
produces:
But clearly it actually is implemented for any lifetime.
The text was updated successfully, but these errors were encountered: