not Send due to await retainment

[Mark-Simulacrum]

• root: updns-0.0.6: start v. end

[pnkfelix]

triage: P-high. Nominating for discussion.

[pnkfelix]

self-assigned for investigation in short term (as in, next two or three days). However I suspect someone from @rust-lang/wg-async-await will be a better choice to investigate this.

[nikomatsakis]

We should try to bisect this

[pnkfelix]

used cargo-bisect-rustc.

Determined:

searched nightlies: from nightly-2019-11-26 to nightly-2020-01-03
regressed nightly: nightly-2019-12-01
searched commits: from 25d8a94 to d8bdb3fdcbd88eb16\
e1a6669236122c41ed2aed3
regressed commit: 9081929

[pnkfelix]

9081929 is a rollup commit of the following PR's:

 - #66379 (Rephrase docs in for ptr)
 - #66589 (Draw vertical lines correctly in compiler error messages)
 - #66613 (Allow customising ty::TraitRef's printing behavior)
 - #66766 (Panic machinery comments and tweaks)
 - #66791 (Handle GlobalCtxt directly from librustc_interface query system)
 - #66793 (Record temporary static references in generator witnesses)
 - #66808 (Cleanup error code)
 - #66826 (Clarifies how to tag users for assigning PRs)
 - #66837 (Clarify `{f32,f64}::EPSILON` docs)
 - #66844 (Miri: do not consider memory allocated by caller_location leaked)
 - #66872 (Minor documentation fix)

I'm guessing this is due to PR #66793

[pnkfelix]

I'm still working on reducing this down to something that I could put into the playpen.

However, one interesting thing I've found locally during my reduction:

This version of the fn proxy will signal an analogous compilation error:

async fn proxy(buf: &[u8]) -> Result<Vec<u8>> {
    let tv = unsafe { TIMEOUT };
    let dur = Duration::from_millis(unsafe { TIMEOUT });
    timeout(Duration::from_millis(unsafe { TIMEOUT }),
            async { Result::Ok(vec![0u8]) })
        .await?
        ;

    Result::Ok(vec![0u8])
}

but if I lift the first argument out to a separate let-binding (i.e. change it to use dur above, or the sub-expression unsafe { TIMEOUT } out to a separate let-binding (i.e. change the argument expression to use tv above), then the error goes away.

I'll admit that I don't know all the details of how the fn-transformation works to support async fn, but I strongly suspect that if this kind of local code change can make an error go away, then that represents a weakness somewhere (either in the fn-transformation, or in the borrowck).

[pnkfelix]

Okay in the details block is a standalone version suitable for the playground:

#![crate_type="lib"]

use std::future::Future;
use std::io::Result;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;

pub fn timeout<T>(_duration: Duration, _future: T) -> Timeout<T>
where
    T: Future,
{
    loop { }
}

#[derive(Debug)]
pub struct Elapsed(());

impl std::fmt::Display for Elapsed {
    fn fmt(&self, w: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(w, "Elapsed")
    }
}

impl std::error::Error for Elapsed {}

impl From<Elapsed> for std::io::Error {
    fn from(_err: Elapsed) -> std::io::Error {
        std::io::ErrorKind::TimedOut.into()
    }
}

pub struct Timeout<T> {
    _value: T,
}

impl<T> Future for Timeout<T>
where
    T: Future,
{
    type Output = std::result::Result<T::Output, Elapsed>;

    fn poll(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Self::Output> {
        loop { }
    }
}

static mut TIMEOUT: u64 = 2000;

use std::marker::PhantomData;

pub struct JoinHandle<T> {
    _p: PhantomData<T>,
}

pub fn spawn<T>(_task: T) -> JoinHandle<T::Output>
where
    T: Future + Send + 'static,
    T::Output: Send + 'static,
{
    loop { }
}

pub async fn my_main() {
    spawn(proxy());
}

// This version of `fn proxy` demonstrates the error
pub async fn proxy() -> Result<Vec<u8>> {
    timeout
        (Duration::from_millis(unsafe { TIMEOUT }),
         async { Result::Ok(vec![0u8]) })
        .await?
        .unwrap()
        ;

    Result::Ok(vec![0u8])
}

// This variant does not generate an error
pub async fn proxy_2() -> Result<Vec<u8>> {
    let dur = Duration::from_millis(unsafe { TIMEOUT });

    timeout
        (dur,
         async { Result::Ok(vec![0u8]) })
        .await?
        .unwrap()
        ;

    Result::Ok(vec![0u8])
}

// This variant does not generate an error
pub async fn proxy_3() -> Result<Vec<u8>> {
    let tv = unsafe { TIMEOUT };

    timeout
        (Duration::from_millis(tv),
         async { Result::Ok(vec![0u8]) })
        .await?
        .unwrap()
        ;

    Result::Ok(vec![0u8])
}

and the error it generates:

error: future cannot be sent between threads safely
  --> src/lib.rs:65:5
   |
56 | pub fn spawn<T>(_task: T) -> JoinHandle<T::Output>
   |        -----
57 | where
58 |     T: Future + Send + 'static,
   |                 ---- required by this bound in `spawn`
...
65 |     spawn(proxy());
   |     ^^^^^ future returned by `proxy` is not `Send`
   |
   = help: within `impl std::future::Future`, the trait `std::marker::Send` is not implemented for `*mut u64`
note: future is not `Send` as this value is used across an await
  --> src/lib.rs:70:5
   |
70 | /     timeout
71 | |         (Duration::from_millis(unsafe { TIMEOUT }),
   | |                                         ------- has type `*mut u64`
72 | |          async { Result::Ok(vec![0u8]) })
73 | |         .await?
   | |______________^ await occurs here, with `TIMEOUT` maybe used later
74 |           .unwrap()
75 |           ;
   |           - `TIMEOUT` is later dropped here

error: aborting due to previous error

[pnkfelix]

So this does seem like somewhat clear (expected?) fallout from PR #67611: a use of a static mut X: T in a generator witness will be recorded as a unsafe pointer *mut T, and then *mut T is never Send, but the spawn method requires that the task you give it is Send.

I believe we can readily update the crate in question to avoid using the static mut in the expression under the RHS of await.

But I do wonder whether we can/should improve the diagnostics here to try to guide the user towards such a solution ...?

[pnkfelix]

I believe we can readily update the crate in question to avoid using the static mut in the expression under the RHS of await.

I might have worded this too strongly, and/or over-reduced my example.

In particular, its easy to move around the access to TIMEOUT: u64. But the borrow of PROXY: Vec<SocketAddr> may be trickier to deal with.

[matthewjasper]

I'll have a look at addressing this with another approach I considered for #66793. If that doesn't work out then I'll have a look at the error message.

[pnkfelix]

I believe we can readily update the crate in question to avoid using the static mut in the expression under the RHS of await.

I might have worded this too strongly, and/or over-reduced my example.

In particular, its easy to move around the access to TIMEOUT: u64. But the borrow of PROXY: Vec<SocketAddr> may be trickier to deal with.

The patch linked is enough to fix the crate in question, but I'm all for @matthewjasper finding an in-compiler solution if we can do it soundly.

https://gist.github.com/pnkfelix/4bdef50d0538fee8f6e0cbc4379f6074

[matthewjasper]

OK. I'm going to write some things up because I've changed my mind a bunch of times about how/if this can be fixed in MIR building and I don't want to keep forgetting things.

Background: auto traits and generators

Generators store values that if their value needs to be preserved across a yield. These values need to be visible for auto-traits, for example a generator may be resumed on a different thread after it yields, and so cannot be Send if any of theses values isn't Send.

There are two passes that determine which values can be included in the generator:

• A pass that runs as part of typeck that walks the HIR to find the types of expressions and then checks for yields in the values scope (as determined by the RegionScopeTree). The types recorded here are stored in the GeneratorWitnesss type contained in the generator, which is then used for auto trait checking. This notably uses the types of expressions that HIR sees.
• The MIR transform that lowers the generator to a state machine. This determines the values that the generator actually stores. As a check for the compiler, we assert that any types that we capture here are included in the witness type (we don't know all auto traits, so can't check only for them).

Background: MIR lowering of statics.

After #66587 statics no longer have a special representation in MIR. They are instead lowered to a dereference of a pointer to the given static:

static X: T = ...;
static mut Y: S = ...;
// ...
use(X);
use(Y);

is lowered as if it were:

static X: T = ...;
static mut Y: S = ...;
const X_PTR: &T = &X; // Imagine that this could compile
const Y_PTR: *mut T = &raw mut Y; // Imagine that this doesn't need an unsafe block
// ...
use(*X_PTR);
use(*Y_PTR);

Which becomes:

_tmp1 = const &X;
use(*_tmp1);
_tmp2 = const &raw mut Y;
use(*_tmp2);

The first problem

The example above is somewhat deceptive in that the temporary that holds the pointer is used immediately after it's assigned. In this case it cannot end up in the generator since it's not live across any user code. However this isn't true in all cases, index and match expressions allow running user code while the pointer is live:

X[EXPR]
// Becomes
_tmp1 = const &X;
_tmp2 = EXPR;
// bounds check
(*_tmp1)[_tmp2]

match Y { a if GUARD => { /* ... */ }
// Becomes
_tmp3 = const &raw mut Y;
// check guard
a = *_tmp3;

In these cases MIR has to put the temporary pointer into the generator state and promptly ICEs.

Fixing the ICE

The obvious solution now is make type checking record the new pointer types in the generator witness in these cases. Unfortunately HIR is not a great tool for specifying "these cases". The methods that the region scope tree provides in HIR are either usually far too large (the "temporary scope") or occasionally too small (the "enclosing scope"). #66793 went for the sound but rather approximate approach, leaving us here.

Other potential fixes

Change MIR building

One idea might be to try changing MIR building so that the temporary is never live across user code. For example, one might change the lowering as follows

X[EXPR]
// Becomes
_tmp2 = EXPR;
// bounds check
_tmp1 = const &X;
(*_tmp1)[_tmp2]

However working out how to do this for match in a reasonable to implement way is harder.

Internal

MIR actually already contains an escape hatch for values that are in the MIR but aren't in the HIR in the form of a flag (called internal) on the variable. We currently use it for drop flags and boxes created by box expressions. It's unclear what exactly the difference is between

static Y = // ...

let x = &Y;
yield;
use(x);
// and
let x = &Y;
yield;
use(&Y);

is. I guess one could argue that auto traits are for layout introspection and this would be further breaking that.

I think that's everything for now.

[tmandry]

Visiting from triage. @matthewjasper were you planning to work on this further?

[tmandry]

It's unclear what exactly the difference is between

static Y = // ...

let x = &Y;
yield;
use(x);
// and
let x = &Y;
yield;
use(&Y);

It's definitely tempting to take advantage of the constraints around statics to just declare these equivalent, and use the internal flag to get out of this. While I don't see any soundness issues around that, I also don't quite trust myself to get this right :)

I guess one could argue that auto traits are for layout introspection and this would be further breaking that.

Can you expand on this? I'm not sure I follow.

[tmandry]

Another solution that comes to mind relies on the fact that these pointers never need to be stored in a generator. They are consts, and the temporaries containing them shouldn't ever be mutated.

We could add semantics for "const temporaries" to MIR, and never store them inside generators. The semantics would (hopefully) be simple: defined once and assigned a value that doesn't depend on any other temporaries; never mutated or borrowed mutably. We can move the statements defining them to the top of the resume function inside the generator transform.

But that would be adding more semantics and special casing to MIR to fix a change which removed extra semantics and special casing, so I'm not sure if that's what we want :)

[matthewjasper]

I guess one could argue that auto traits are for layout introspection and this would be further breaking that.

Can you expand on this? I'm not sure I follow.

The argument is that the given the following auto trait we guarantee that if T: DoesntContainRawMutPtr then there are no raw pointers directly stored in an instance of T.

auto trait DoesntContainRawMutPtr {}

impl<T: ?Sized> !DoesntContainRawMutPtr for *mut T {}
impl<T: ?Sized> DoesntContainRawMutPtr for *const T {}
impl<T: ?Sized> DoesntContainRawMutPtr for &T {}
impl<T: ?Sized> DoesntContainRawMutPtr for &mut T {}

If we make that guarantee then using internal in generators would not be OK. I would argue that we're not making such (or really any) guarantees with user defined auto traits.

[tmandry]

Yeah, I don't think auto traits should expose layout implementation details of generators.

And this really does seem like a pure implementation detail -- it doesn't have any bearing on what kinds of (unsafe) operations are allowed on a generator, like a self-reference would.

cc @rust-lang/lang

[nikomatsakis]

So, I've been putting off commenting here to let these ideas sit, but let me leave a comment. I think I've reached a conclusion.

TL;DR. I think we should mark these *mut types as internal, but I think there are some better options for fixing this in the medium term, and this interaction should be taken into account as part of the "MIR 2.0" effort to rework places.

Auto traits weren't really intended as a tool for probing layout, but rather semantic considerations. This is why traits like Send can be implemented by users. So I don't really think that auto traits give any sort of guarantee about their layout.

Rather, this question is about is simply "what are the constituent types exposed by a generator", and this should correspond to the types that user code can directly access. The raw pointers here are not directly accessible. The values that are accessible are the "live values" that the user's code may later use -- and of course the code currently makes the constituent types be a superset of those, based on the AST structure.

Of course, the real root cause of this error is a simplifying change to the definition of Place in MIR. It used to be that a Place had this grammar:

Place = PlaceBase | Place . field | * Place | Place [ Expr ] | ...
PlaceBase = Static | Local

but now we changed PlaceBase to just Local, which is why we are required to introduce the new temporary (and Local that holds the address of the Static).

As we've been talking about this change, I've been saying that we should be considering the older definition of Place as the one that is used by various analyses (and in particular by borrow check). This particular transformation of removing statics doesn't affect borrow check, but we failed to take into account the fact that the set of live variables can effect the definition of what is live in a generator.

In particular, compiling E[5] for some expression E requires a temporary unless E corresponds to a place -- those places used to include statics like X, but they no longer do. But I think that for analysis purposes, it should appear as if they do.

We could fix this in two ways. We could write some code motion / duplication that takes temporaries like X = &raw mut Y that are live across a yield and creates a fresh variable after the yield. The idea would be to make them never live across a yield. We can do this because that &raw mut Y operation is a compile-time constant, really, and thus the value doesn't need to be live across a yield. This might be worth doing as a size optimization regardless.

Secondly, some of the proposals we've been kicking around for going further with simplifying Place include the idea of a distinct sort of temporaries that explicitly represent a "partially constructed Place that hasn't been used by the user". The idea was to allow borrowck (and other analyses) to trivially reconstruct the Place. We might want to add this interaction with auto-traits into the consideration for that design, and maybe it takes us some place (e.g., these special temporaries are not considered live for the purposes of auto traits).

Having written out both of those options, though, I sort of lean towards the first, at least for this specific issue.

[Zoxc]

I think these references and raw pointers created by MIR construction should be marked as internal. They are not visible at the language level. They can be viewed as just prefetching the address of a static, so the compiler could pick any type it likes for it (*const T, *mut T, usize, some compiler internal type) so it must not affect auto trait inference.

Drop flags are ignored for similar reasons. You cannot use an auto trait like this to detect them:

auto trait DoesntContainDropFlags {}
impl !DoesntContainRawMutPtr for bool {}

So I think that #66793 should be reverted and the temporary static references should be marked as internal.

[nikomatsakis]

Note: @matthewjasper opened #68494 instituting this change and I r+'d it, since

• fixing the regression seems pretty important and time is of the essence;
• there is general consensus here on the thread;
• this is only restoring older behavior.

In particular, there were some suggestions that this might be a lang team issue; I think there is some 'intersection' and I don't want the perception that I'm trying to shortcircuit discussion. That said, I also think this is primarily a compiler team matter, since it's a deviation from the old behavior, and further this deviation was caused by altering the MIR away from the "natural, Rust-based" definition and towards a lower-level definition.