impl Future used across crate boundaries produces an error

[Limeth]

Compilation of crate docs (at the beginning of lib.rs) failed when running env RUST_BACKTRACE=full cargo test -- --nocapture.

I tried this code:

//! An API for http://ip-api.com/ written in the Rust language.
//!
//! This library lets you request information about an IP address.
//! It uses `futures` to deliver the result.
//!
//! You will need the `tokio_core` create in order to use this library.
//! # Examples
//!
//! ```
//! # extern crate futures;
//! # extern crate tokio_core;
//! # extern crate ip_api;
//! use std::net::{IpAddr, Ipv4Addr};
//! use tokio_core::reactor::Core;
//! use futures::future::Future;
//! use ip_api::IpApi;
//!
//! # #[allow(unused_variables)]
//! # fn main() {
//! let mut core = Core::new().unwrap();
//! let handle = core.handle();
//! let ip_api = IpApi::new(handle);
//! let future = ip_api.request(Some(IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8))))
//!     .map(|result| {
//!         println!("{:?}", result);
//!     });
//!
//! core.run(future).unwrap();
//! # }
//! ```

(see lib.rs for the whole file, successfully compiled)

I expected to see this happen: A successful compilation, or an error message displayed.

Meta

rustc 1.18.0-nightly (c58c928e6 2017-04-11)
binary: rustc
commit-hash: c58c928e658d2e45f816fd05796a964aa83759da
commit-date: 2017-04-11
host: x86_64-unknown-linux-gnu
release: 1.18.0-nightly
LLVM version: 3.9

Log

    Finished dev [unoptimized + debuginfo] target(s) in 5.55 secs
     Running target/debug/deps/ip_api-b628445a6205873b

running 1 test
test tests::it_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests ip-api

running 2 tests
error: internal compiler error: /checkout/src/librustc_trans/collector.rs:668: Cannot create local trans-item for DefId { krate: CrateNum(26), node: DefIndex(23) => ip_api/b6a065cc65916384c1a936b4d0f34bb4::get_string[0] }

thread 'rustc' panicked at 'Box<Any>', /checkout/src/librustc_errors/lib.rs:416
stack backtrace:
   0:     0x7f0464cff0f3 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hab274209b3900f9c
                               at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1:     0x7f0464cf97d4 - std::sys_common::backtrace::_print::h8f655fc4b25b9b70
                               at /checkout/src/libstd/sys_common/backtrace.rs:71
   2:     0x7f0464d0d047 - std::panicking::default_hook::{{closure}}::hd0f0fde26cdd4a91
                               at /checkout/src/libstd/sys_common/backtrace.rs:60
                               at /checkout/src/libstd/panicking.rs:355
   3:     0x7f0464d0cbac - std::panicking::default_hook::h123df66825ae8c79
                               at /checkout/src/libstd/panicking.rs:365
   4:     0x7f0464d0d51b - std::panicking::rust_panic_with_hook::h3635757261b59272
                               at /checkout/src/libstd/panicking.rs:549
   5:     0x7f045d7f2e27 - std::panicking::begin_panic::h7bb0c7bba9a436bf
   6:     0x7f045d80cb79 - rustc_errors::Handler::bug::h45cc5a767a46e974
   7:     0x7f0461c0138a - rustc::session::opt_span_bug_fmt::{{closure}}::hfbfe9f66fec7d051
   8:     0x7f0461c00dc7 - rustc::session::opt_span_bug_fmt::h4aac6d7f7dbc3512
   9:     0x7f0461c00a22 - rustc::session::bug_fmt::h0bea60fd591127eb
  10:     0x7f0463a51de1 - rustc_trans::collector::should_trans_locally::h4656782e9eb00175
  11:     0x7f0463a51a5b - rustc_trans::collector::visit_instance_use::h4f153b416289b87f
  12:     0x7f0463a5173d - <rustc_trans::collector::MirNeighborCollector<'a, 'tcx> as rustc::mir::visit::Visitor<'tcx>>::visit_terminator_kind::h101e5aee6ff39360
  13:     0x7f04639e6b88 - rustc::mir::visit::Visitor::visit_mir::hcbffa68f7c70641d
  14:     0x7f0463a534b8 - rustc_trans::collector::collect_neighbours::hd2f5dd549eec023d
  15:     0x7f0463a4faef - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  16:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  17:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  18:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  19:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  20:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  21:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  22:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  23:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  24:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  25:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  26:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  27:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  28:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  29:     0x7f0463a4fdcf - rustc_trans::collector::collect_items_rec::h6c7176696c5743fe
  30:     0x7f0463a4c526 - rustc_trans::base::collect_and_partition_translation_items::{{closure}}::h63d2a00c3084c991
  31:     0x7f0463a47970 - rustc_trans::base::collect_and_partition_translation_items::hadb2691f60818418
  32:     0x7f0463a3bcbb - rustc_trans::base::trans_crate::hace7d592cac048a9
  33:     0x7f046461d1c9 - rustc_driver::driver::phase_4_translate_to_llvm::h904e3fc572f42e2d
  34:     0x7f04645db2b8 - rustc_driver::driver::compile_input::{{closure}}::hc476f85d4e1ed8fa
  35:     0x7f046460fbf5 - rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}::ha83538a03579857c
  36:     0x7f046454a4a2 - rustc::ty::context::TyCtxt::create_and_enter::hd9482ac47b3dbb7c
  37:     0x7f0464603b2b - rustc_driver::driver::phase_3_run_analysis_passes::hfea44285662197cc
  38:     0x7f04645d11ef - rustc_driver::driver::compile_input::hc74994fea2a7b3d8
  39:     0x7f0465030893 - std::panicking::try::do_call::he2b177adc905cdbd
  40:     0x7f0464d1654a - __rust_maybe_catch_panic
                               at /checkout/src/libpanic_unwind/lib.rs:98
  41:     0x7f04651aa01b - rustdoc::test::runtest::h0596586cc6ee1092
  42:     0x7f046502ff97 - std::panicking::try::do_call::h5e36b666e0a8d2a1
  43:     0x7f0464d1654a - __rust_maybe_catch_panic
                               at /checkout/src/libpanic_unwind/lib.rs:98
  44:     0x7f046507a594 - <F as alloc::boxed::FnBox<A>>::call_box::h161bb4f304a420a4
  45:     0x7f0464d0be14 - std::sys::imp::thread::Thread::new::thread_start::h1b316ba1f093c5a7
                               at /checkout/src/liballoc/boxed.rs:650
                               at /checkout/src/libstd/sys_common/thread.rs:21
                               at /checkout/src/libstd/sys/unix/thread.rs:84
  46:     0x7f045cd516b9 - start_thread
  47:     0x7f04649ba82c - clone
  48:                0x0 - <unknown>
thread 'rustc' panicked at 'couldn't compile the test', /checkout/src/librustdoc/test.rs:270
stack backtrace:
   0:     0x7f0464cff0f3 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hab274209b3900f9c
                               at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1:     0x7f0464cf97d4 - std::sys_common::backtrace::_print::h8f655fc4b25b9b70
                               at /checkout/src/libstd/sys_common/backtrace.rs:71
   2:     0x7f0464d0d047 - std::panicking::default_hook::{{closure}}::hd0f0fde26cdd4a91
                               at /checkout/src/libstd/sys_common/backtrace.rs:60
                               at /checkout/src/libstd/panicking.rs:355
   3:     0x7f0464d0cbac - std::panicking::default_hook::h123df66825ae8c79
                               at /checkout/src/libstd/panicking.rs:365
   4:     0x7f0464d0d51b - std::panicking::rust_panic_with_hook::h3635757261b59272
                               at /checkout/src/libstd/panicking.rs:549
   5:     0x7f046502fc6f - std::panicking::begin_panic::h6a220d6e24d23e61
   6:     0x7f04651ab858 - rustdoc::test::runtest::h0596586cc6ee1092
   7:     0x7f046502ff97 - std::panicking::try::do_call::h5e36b666e0a8d2a1
   8:     0x7f0464d1654a - __rust_maybe_catch_panic
                               at /checkout/src/libpanic_unwind/lib.rs:98
   9:     0x7f046507a594 - <F as alloc::boxed::FnBox<A>>::call_box::h161bb4f304a420a4
  10:     0x7f0464d0be14 - std::sys::imp::thread::Thread::new::thread_start::h1b316ba1f093c5a7
                               at /checkout/src/liballoc/boxed.rs:650
                               at /checkout/src/libstd/sys_common/thread.rs:21
                               at /checkout/src/libstd/sys/unix/thread.rs:84
  11:     0x7f045cd516b9 - start_thread
  12:     0x7f04649ba82c - clone
  13:                0x0 - <unknown>
test src/lib.rs -  (line 9) ... FAILED
test src/lib.rs - IpApi (line 92) ... ok

failures:

failures:
    src/lib.rs -  (line 9)

test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured

error: test failed, to rerun pass '--doc'

Edit: Minimal case by @alexcrichton

// foo.rs
#![crate_type = "lib"]
#![feature(conservative_impl_trait)]

fn bar() {}

pub fn foo() -> impl FnMut() {
    || {
        bar()
    }
}

// bar.rs
extern crate foo;

fn main() {
    foo::foo()();
}

producer the error when built using

$ rustc +nightly foo.rs
$ rustc +nightly bar.rs -L .

[nateozem]

Use no_run to opt out doc test because in reality, when you execute cargo test, rustdoc is also executed with the flag of --test

$ cargo rustdoc -- --test -L target/debug

Please correct me if I'm wrong.

So add no-run as so and it should run fine.

//! ```no_run
//! // ..... <doc code here>
//! ```

However, I wonder if this is still an issue to be fix?
It'd be great if the compiler annotate to use no_run if we don't want the documented code to be tested.

Also to note, I believe the reason for the error because doc code failed. If it passed, this error wouldn't be shown.

[nateozem]

Here is a link that reference to what I just mention about no_run. It has other useful information too.
https://doc.rust-lang.org/book/documentation.html#running-documentation-tests

[Limeth]

I don't want to disable the tests, though. I do like that it checks whether the code in the doc works.

[steveklabnik]

However, I wonder if this is still an issue to be fix?

Yes, it is, because an ICE is always a bug.

[arielb1]

Looks like the "standard" impl-trait reachability problems - the ICE occurs because to_string isn't marked as reachable.

[tomaka]

Does someone have an idea how I could bypass this ICE? (I have it in regular code, not in a doctest).

I've been trying various work-arounds for 40 minutes now and am totally blocked by it.

[jonhoo]

Same thing happens if you comment out no_run from https://github.com/jonhoo/fantoccini/blob/600fff600bf19a3aa3266540d11602d3e6f5a66e/src/lib.rs#L123. EDIT: In fact, any crate compiled against the futures branch of the fantoccini crate linked above seem to encounter this ICE.

[jonhoo]

@Limeth @arielb1 I believe this is not just about doctests, but happens occasionally when impl Future is used across crate boundaries. The shortest replicating example I have been able to come up with involves two crates, foo and bar, like this:

// foo/src/lib.rs
#![feature(conservative_impl_trait)]
extern crate futures;
use futures::{Future, future};

pub struct Foo;
impl Foo {
    fn id(self) -> Self {
        self
    }

    pub fn new() -> impl Future<Item = Self, Error = ()> + 'static {
        future::lazy(|| future::ok(Foo.id()))
    }
}

// bar/src/main.rs
extern crate foo;
extern crate tokio_core;

fn main() {
    let mut core = tokio_core::reactor::Core::new().unwrap();
    core.run(foo::Foo::new()).unwrap();
}

$ cargo run # in bar/
error: internal compiler error: /checkout/src/librustc_trans/collector.rs:657: Cannot create local trans-item for DefId { krate: CrateNum(11), node: DefIndex(9) => foo/c66423d::{{impl}}[0]::id[0] }

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.20.0-nightly (445077963 2017-06-20) running on x86_64-unknown-linux-gnu

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'Box<Any>', /checkout/src/librustc_errors/lib.rs:478
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at /checkout/src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   3: std::panicking::default_hook
             at /checkout/src/libstd/panicking.rs:365
   4: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:549
   5: std::panicking::begin_panic
   6: rustc_errors::Handler::bug
   7: rustc::session::opt_span_bug_fmt::{{closure}}
   8: rustc::session::opt_span_bug_fmt
   9: rustc::session::bug_fmt
  10: rustc_trans::collector::should_trans_locally
  11: rustc_trans::collector::visit_instance_use
  12: <rustc_trans::collector::MirNeighborCollector<'a, 'tcx> as rustc::mir::visit::Visitor<'tcx>>::visit_terminator_kind
  13: rustc::mir::visit::Visitor::visit_mir
  14: rustc_trans::collector::collect_items_rec
  15: rustc_trans::collector::collect_items_rec
  16: rustc_trans::collector::collect_items_rec
  17: rustc_trans::collector::collect_items_rec
  18: rustc_trans::collector::collect_items_rec
  19: rustc_trans::collector::collect_items_rec
  20: rustc_trans::collector::collect_items_rec
  21: rustc_trans::collector::collect_items_rec
  22: rustc_trans::collector::collect_items_rec
  23: rustc_trans::collector::collect_items_rec
  24: rustc_trans::collector::collect_items_rec
  25: rustc_trans::collector::collect_items_rec
  26: rustc_trans::collector::collect_items_rec
  27: rustc_trans::base::collect_and_partition_translation_items::{{closure}}
  28: rustc_trans::base::trans_crate
  29: rustc_driver::driver::phase_4_translate_to_llvm
  30: rustc_driver::driver::compile_input::{{closure}}
  31: rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}
  32: rustc_driver::driver::phase_3_run_analysis_passes
  33: rustc_driver::driver::compile_input
  34: rustc_driver::run_compiler

error: Could not compile `bar`.

[Limeth]

Thank you for the thorough examination.

[jonhoo]

@Limeth happy to help! You may want to update the issue title though?

[jonhoo]

For convenience, the tracking issue for conservative_impl_trait is #34511.

[nikomatsakis]

Do we have a reproducible, relatively minimal test case here? Seems like that is the first goal.

[nikomatsakis]

triage: P-medium

[nikomatsakis]

triage: P-high

we do after all want to stabilize impl trait and use it for exactly this use case =)

[jonhoo]

@nikomatsakis the smallest code example I could come up with to reproduce is the one in #41297 (comment), but it unfortunately pulls in both futures and tokio :/

[eddyb]

@jonhoo I think it's enough, because, remember, from the compiler's point of view, futures are just stranger iterators. Here it looks like necessary elements involve a monomorphic closure calling a monomorphic private function. What happens if you put #[inline] on the id method or make it generic on a type? I don't even know why it's trying to translate that function across crates.

[jonhoo]

@eddyb Both #[inline] on id and making id generic fixes the problem.

[ishitatsuyuki]

#43135 is a duplicate, and has a test case described there; can someone work on this?

[alexcrichton]

I have what I believe is a minimal case for this:

// foo.rs
#![crate_type = "lib"]
#![feature(conservative_impl_trait)]

fn bar() {}

pub fn foo() -> impl FnMut() {
    || {
        bar()
    }
}

// bar.rs
extern crate foo;

fn main() {
    foo::foo()();
}

$ rustc +nightly foo.rs
$ rustc +nightly bar.rs -L .
error: internal compiler error: /checkout/src/librustc_trans/collector.rs:735: Cannot create local trans-item for DefId { krate: CrateNum(12), node: DefIndex(3) => foo/8cd878b::bar[0] }

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.21.0-nightly (215e0b10e 2017-08-08) running on x86_64-unknown-linux-gnu

thread 'rustc' panicked at 'Box<Any>', /checkout/src/librustc_errors/lib.rs:486:8
note: Run with `RUST_BACKTRACE=1` for a backtrace.

[ishitatsuyuki]

This should be E-needstest. I confirmed the fix for my code, and I assume this is resolved.

Ref: #43857

[Arnavion]

Not sure it's completely resolved.

I have a lib crate A and a bin crate B that uses A. A has two functions that hit this - fn foo() -> Box<Future<...>> and fn bar() -> Box<Stream<...>>. Initially A took 12s to compile and B took 15s to compile.

When I changed foo to return impl Future, A still took 12s to compile but B rose to 35s.

When I also converted bar to return impl Stream, A took 15s to compile and B kept compiling for more than 5 minutes before I aborted it. I could see rustc maxing out one core throughout that time.

This is with 2017-08-15's nightly so it has #43857

[ishitatsuyuki]

Sure it is very slow, and probably a separate issue. I think #43787 is related, but anyway this issue should be kept to track only the ICE.

[Arnavion]

I tried reverting the two PRs mentioned in #43787 but ran into conflicts. Then I changed the one line suggested there and this time B finished building after ~20 minutes. So yes, it looks like #43787 is getting hit when I change the two functions to return impl trait.

[ishitatsuyuki]

I'm not sure if your comparison makes sense. I have totally no confidence if #43787 is related or not, and "aborting after 5min" and "completed in 20min" sounds same to me.

[Arnavion]

"Aborting after 5min" was the experiment from 6 hours ago. After I confirmed that the local build finished compiling in 20 mins, I reran the build using 2017-08-15 nightly and it's still running as of 40 mins. So yes, changing the one line described in the comment in #43787 did have an effect. It's not a complete fix (it shouldn't be taking 20 mins to compile) which is why I said it's some combination of the slowdown + this impl trait issue.

[ishitatsuyuki]

I think giving a reproduce case (not necessarily short) to the compiler team would help as they would profile and narrow the problem down. Do you have a specific revision in public repository?

[Arnavion]

https://github.com/Arnavion/fac-rs/commits/master - compiles
https://github.com/Arnavion/fac-rs/commits/rust-41297 - never finishes compiling

Edit: Just to quantify "never finishes compiling" - it didn't finish even after being left for 6 hours overnight.

[mhristache]

@Arnavion can you run with RUST_LOG=trace and paste the line where it starts hanging?

[mhristache]

I had a similar issue. See #40839 (comment)

[nikomatsakis]

@Arnavion can you open a separate issue perhaps on that problem?

[nikomatsakis]

Closing for now, since the main problem here is fixed. @Arnavion if you can open a separate issue with a complete description of the problems you encountered, would be much appreciated!

[Arnavion]

@nikomatsakis Not needed. #43999 already fixes it.