Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix overconstrained Send impls in btree internals #102680

Merged
merged 2 commits into from
Oct 6, 2022

Conversation

dtolnay
Copy link
Member

@dtolnay dtolnay commented Oct 4, 2022

Fixes dtolnay/async-trait#215.

Minimal repro:

use std::collections::btree_map::Iter;

fn require_send<T: Send>(_: T) {}

fn main() {
    require_send(async {
        let _iter = None::<Iter<(), &()>>;
        async {}.await;
    });
}
error: higher-ranked lifetime error
 --> src/main.rs:6:5
  |
6 | /     require_send(async {
7 | |         let _iter = None::<Iter<(), &()>>;
8 | |         async {}.await;
9 | |     });
  | |______^
  |
  = note: could not prove `impl Future<Output = ()>: Send`

Not-quite-so-minimal repro:

use std::collections::BTreeMap;
use std::future::Future;

fn spawn<T: Future + Send>(_: T) {}

async fn f() {
    let map = BTreeMap::<u32, Box<dyn Send + Sync>>::new();
    for _ in &map {
        async {}.await;
    }
}

fn main() {
    spawn(f());
}
error: higher-ranked lifetime error
  --> src/main.rs:14:5
   |
14 |     spawn(f());
   |     ^^^^^^^^^^
   |
   = note: could not prove `impl Future<Output = ()>: Send`

I am not familiar with the btree internals, but it seems clear to me that the async fn f above should return a Send future. Using HashMap instead of BTreeMap in that code makes it already return a Send future.

The "higher-ranked lifetime error" message may be a regression in Rust 1.63. Using older compilers the error message was more detailed:

error: implementation of `Send` is not general enough
  --> src/main.rs:14:5
   |
14 |     spawn(f());
   |     ^^^^^ implementation of `Send` is not general enough
   |
   = note: `Send` would have to be implemented for the type `alloc::collections::btree::node::NodeRef<alloc::collections::btree::node::marker::Immut<'0>, u32, Box<(dyn Send + Sync + '1)>, alloc::collections::btree::node::marker::LeafOrInternal>`, for any two lifetimes `'0` and `'1`...
   = note: ...but `Send` is actually implemented for the type `alloc::collections::btree::node::NodeRef<alloc::collections::btree::node::marker::Immut<'2>, u32, Box<dyn Send + Sync>, alloc::collections::btree::node::marker::LeafOrInternal>`, for some specific lifetime `'2`

error: implementation of `Send` is not general enough
  --> src/main.rs:14:5
   |
14 |     spawn(f());
   |     ^^^^^ implementation of `Send` is not general enough
   |
   = note: `Send` would have to be implemented for the type `alloc::collections::btree::node::NodeRef<alloc::collections::btree::node::marker::Immut<'0>, u32, Box<(dyn Send + Sync + '1)>, alloc::collections::btree::node::marker::Leaf>`, for any two lifetimes `'0` and `'1`...
   = note: ...but `Send` is actually implemented for the type `alloc::collections::btree::node::NodeRef<alloc::collections::btree::node::marker::Immut<'2>, u32, Box<dyn Send + Sync>, alloc::collections::btree::node::marker::Leaf>`, for some specific lifetime `'2`

@rust-highfive
Copy link
Collaborator

r? @thomcc

(rust-highfive has picked a reviewer for you, use r? to override)

@rustbot
Copy link
Collaborator

rustbot commented Oct 4, 2022

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@rustbot rustbot added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Oct 4, 2022
@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 4, 2022
@dtolnay dtolnay removed the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Oct 4, 2022
@Aaron1011
Copy link
Member

The general issue is #64552 - this fixes a particular instance by eliminating the need for rustc to attempt to prove a lifetime bound on the erased witness types.

As long as the new impl is sound, I think it's fine to merge this. However, users will continue to see these unhelpful errors in other cases until the underlying issue is resolved.

Copy link
Member

@thomcc thomcc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, good catch. It doesn't make sense that we'd bound those on the same lifetime we use in the marker, so this looks fine to me.

I don't think this needs libs-api sign-off, as it's hard to imagine how this could be intentionally forbidden, but you'd have a better idea (as someone actually part of that team) so I'll leave that choice to you.

r=me if you move the tests out of uitest (or feel strongly that they shouldn't be moved).

@@ -0,0 +1,20 @@
// check-fail
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most people don't bother to run the uitests locally for library patches, and the error message is the same as any other Sync error, so IMO its a bit better to do this as a compile_fail doctest on something private, and put check-pass in the normal tests.

That said this is unlikely to unexpectedly break in the future, so if you feel strongly that they should be a uitest, fine by me as-is.

Currently pretty much all of the btree_map and btree_set ones fail, as
well as linked_list::DrainFilter.

    error: higher-ranked lifetime error
      --> library/alloc/tests/autotraits.rs:38:5
       |
    38 | /     require_send_sync(async {
    39 | |         let _v = None::<alloc::collections::btree_map::Iter<'_, &u32, &u32>>;
    40 | |         async {}.await;
    41 | |     });
       | |______^
       |
       = note: could not prove `impl Future<Output = ()>: Send`

    error: implementation of `Send` is not general enough
      --> library/alloc/tests/autotraits.rs:56:5
       |
    56 | /     require_send_sync(async {
    57 | |         let _v = None::<
    58 | |             alloc::collections::btree_map::DrainFilter<
    59 | |                 '_,
    ...  |
    65 | |         async {}.await;
    66 | |     });
       | |______^ implementation of `Send` is not general enough
       |
       = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
       = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: implementation of `Send` is not general enough
      --> library/alloc/tests/autotraits.rs:68:5
       |
    68 | /     require_send_sync(async {
    69 | |         let _v = None::<alloc::collections::btree_map::Entry<'_, &u32, &u32>>;
    70 | |         async {}.await;
    71 | |     });
       | |______^ implementation of `Send` is not general enough
       |
       = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
       = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: higher-ranked lifetime error
      --> library/alloc/tests/autotraits.rs:88:5
       |
    88 | /     require_send_sync(async {
    89 | |         let _v = None::<alloc::collections::btree_map::Iter<'_, &u32, &u32>>;
    90 | |         async {}.await;
    91 | |     });
       | |______^
       |
       = note: could not prove `impl Future<Output = ()>: Send`

    error: implementation of `Send` is not general enough
      --> library/alloc/tests/autotraits.rs:93:5
       |
    93 | /     require_send_sync(async {
    94 | |         let _v = None::<alloc::collections::btree_map::IterMut<'_, &u32, &u32>>;
    95 | |         async {}.await;
    96 | |     });
       | |______^ implementation of `Send` is not general enough
       |
       = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
       = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:98:5
        |
    98  | /     require_send_sync(async {
    99  | |         let _v = None::<alloc::collections::btree_map::Keys<'_, &u32, &u32>>;
    100 | |         async {}.await;
    101 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: implementation of `Send` is not general enough
       --> library/alloc/tests/autotraits.rs:103:5
        |
    103 | /     require_send_sync(async {
    104 | |         let _v = None::<alloc::collections::btree_map::OccupiedEntry<'_, &u32, &u32>>;
    105 | |         async {}.await;
    106 | |     });
        | |______^ implementation of `Send` is not general enough
        |
        = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
        = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: implementation of `Send` is not general enough
       --> library/alloc/tests/autotraits.rs:108:5
        |
    108 | /     require_send_sync(async {
    109 | |         let _v = None::<alloc::collections::btree_map::OccupiedError<'_, &u32, &u32>>;
    110 | |         async {}.await;
    111 | |     });
        | |______^ implementation of `Send` is not general enough
        |
        = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
        = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:113:5
        |
    113 | /     require_send_sync(async {
    114 | |         let _v = None::<alloc::collections::btree_map::Range<'_, &u32, &u32>>;
    115 | |         async {}.await;
    116 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: implementation of `Send` is not general enough
       --> library/alloc/tests/autotraits.rs:118:5
        |
    118 | /     require_send_sync(async {
    119 | |         let _v = None::<alloc::collections::btree_map::RangeMut<'_, &u32, &u32>>;
    120 | |         async {}.await;
    121 | |     });
        | |______^ implementation of `Send` is not general enough
        |
        = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
        = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: implementation of `Send` is not general enough
       --> library/alloc/tests/autotraits.rs:123:5
        |
    123 | /     require_send_sync(async {
    124 | |         let _v = None::<alloc::collections::btree_map::VacantEntry<'_, &u32, &u32>>;
    125 | |         async {}.await;
    126 | |     });
        | |______^ implementation of `Send` is not general enough
        |
        = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
        = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:128:5
        |
    128 | /     require_send_sync(async {
    129 | |         let _v = None::<alloc::collections::btree_map::Values<'_, &u32, &u32>>;
    130 | |         async {}.await;
    131 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: implementation of `Send` is not general enough
       --> library/alloc/tests/autotraits.rs:133:5
        |
    133 | /     require_send_sync(async {
    134 | |         let _v = None::<alloc::collections::btree_map::ValuesMut<'_, &u32, &u32>>;
    135 | |         async {}.await;
    136 | |     });
        | |______^ implementation of `Send` is not general enough
        |
        = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
        = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:146:5
        |
    146 | /     require_send_sync(async {
    147 | |         let _v = None::<alloc::collections::btree_set::Difference<'_, &u32>>;
    148 | |         async {}.await;
    149 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: implementation of `Send` is not general enough
       --> library/alloc/tests/autotraits.rs:151:5
        |
    151 | /     require_send_sync(async {
    152 | |         let _v = None::<alloc::collections::btree_set::DrainFilter<'_, &u32, fn(&&u32) -> bool>>;
    153 | |         async {}.await;
    154 | |     });
        | |______^ implementation of `Send` is not general enough
        |
        = note: `Send` would have to be implemented for the type `&'0 u32`, for any lifetime `'0`...
        = note: ...but `Send` is actually implemented for the type `&'1 u32`, for some specific lifetime `'1`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:156:5
        |
    156 | /     require_send_sync(async {
    157 | |         let _v = None::<alloc::collections::btree_set::Intersection<'_, &u32>>;
    158 | |         async {}.await;
    159 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:166:5
        |
    166 | /     require_send_sync(async {
    167 | |         let _v = None::<alloc::collections::btree_set::Iter<'_, &u32>>;
    168 | |         async {}.await;
    169 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:171:5
        |
    171 | /     require_send_sync(async {
    172 | |         let _v = None::<alloc::collections::btree_set::Range<'_, &u32>>;
    173 | |         async {}.await;
    174 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:176:5
        |
    176 | /     require_send_sync(async {
    177 | |         let _v = None::<alloc::collections::btree_set::SymmetricDifference<'_, &u32>>;
    178 | |         async {}.await;
    179 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: higher-ranked lifetime error
       --> library/alloc/tests/autotraits.rs:181:5
        |
    181 | /     require_send_sync(async {
    182 | |         let _v = None::<alloc::collections::btree_set::Union<'_, &u32>>;
    183 | |         async {}.await;
    184 | |     });
        | |______^
        |
        = note: could not prove `impl Future<Output = ()>: Send`

    error: future cannot be sent between threads safely
       --> library/alloc/tests/autotraits.rs:243:23
        |
    243 |       require_send_sync(async {
        |  _______________________^
    244 | |         let _v =
    245 | |             None::<alloc::collections::linked_list::DrainFilter<'_, &u32, fn(&mut &u32) -> bool>>;
    246 | |         async {}.await;
    247 | |     });
        | |_____^ future created by async block is not `Send`
        |
        = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `NonNull<std::collections::linked_list::Node<&u32>>`
    note: future is not `Send` as this value is used across an await
       --> library/alloc/tests/autotraits.rs:246:17
        |
    244 |         let _v =
        |             -- has type `Option<std::collections::linked_list::DrainFilter<'_, &u32, for<'a, 'b> fn(&'a mut &'b u32) -> bool>>` which is not `Send`
    245 |             None::<alloc::collections::linked_list::DrainFilter<'_, &u32, fn(&mut &u32) -> bool>>;
    246 |         async {}.await;
        |                 ^^^^^^ await occurs here, with `_v` maybe used later
    247 |     });
        |     - `_v` is later dropped here
    note: required by a bound in `require_send_sync`
       --> library/alloc/tests/autotraits.rs:3:25
        |
    3   | fn require_send_sync<T: Send + Sync>(_: T) {}
        |                         ^^^^ required by this bound in `require_send_sync`

    error: future cannot be shared between threads safely
       --> library/alloc/tests/autotraits.rs:243:23
        |
    243 |       require_send_sync(async {
        |  _______________________^
    244 | |         let _v =
    245 | |             None::<alloc::collections::linked_list::DrainFilter<'_, &u32, fn(&mut &u32) -> bool>>;
    246 | |         async {}.await;
    247 | |     });
        | |_____^ future created by async block is not `Sync`
        |
        = help: within `impl Future<Output = ()>`, the trait `Sync` is not implemented for `NonNull<std::collections::linked_list::Node<&u32>>`
    note: future is not `Sync` as this value is used across an await
       --> library/alloc/tests/autotraits.rs:246:17
        |
    244 |         let _v =
        |             -- has type `Option<std::collections::linked_list::DrainFilter<'_, &u32, for<'a, 'b> fn(&'a mut &'b u32) -> bool>>` which is not `Sync`
    245 |             None::<alloc::collections::linked_list::DrainFilter<'_, &u32, fn(&mut &u32) -> bool>>;
    246 |         async {}.await;
        |                 ^^^^^^ await occurs here, with `_v` maybe used later
    247 |     });
        |     - `_v` is later dropped here
    note: required by a bound in `require_send_sync`
       --> library/alloc/tests/autotraits.rs:3:32
        |
    3   | fn require_send_sync<T: Send + Sync>(_: T) {}
        |                                ^^^^ required by this bound in `require_send_sync`
@dtolnay
Copy link
Member Author

dtolnay commented Oct 5, 2022

Moved test to library/alloc/tests/autotraits.rs and extended it to cover alloc::collections more thoroughly, which revealed a linked_list::DrainFilter bug #43244 (comment).

I left out negative tests for now but I am thinking of following up with some based on https://github.com/dtolnay/proc-macro2/blob/1.0.46/tests/marker.rs or the static_assertions crate.

@thomcc
Copy link
Member

thomcc commented Oct 6, 2022

@bors r+

@bors
Copy link
Contributor

bors commented Oct 6, 2022

📌 Commit 4fdd0d9 has been approved by thomcc

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 6, 2022
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 6, 2022
…iaskrgr

Rollup of 5 pull requests

Successful merges:

 - rust-lang#98496 (make `compare_const_impl` a query and use it in `instance.rs`)
 - rust-lang#102680 (Fix overconstrained Send impls in btree internals)
 - rust-lang#102718 (Fix `opaque_hidden_inferred_bound` lint ICE)
 - rust-lang#102725 (Remove `-Ztime`)
 - rust-lang#102736 (Migrate search input color to CSS variable)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit e77e020 into rust-lang:master Oct 6, 2022
@rustbot rustbot added this to the 1.66.0 milestone Oct 6, 2022
@dtolnay dtolnay deleted the btreesend branch October 24, 2023 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

higher-ranked lifetime error with a simple example involving a BTreeMap
6 participants