Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo test compiles twice #851

Closed
nixpulvis opened this issue Nov 12, 2014 · 34 comments
Closed

cargo test compiles twice #851

nixpulvis opened this issue Nov 12, 2014 · 34 comments

Comments

@nixpulvis
Copy link

Running cargo test compiles the code twice before running the compiled tester binary. This is obviously irritating to look at when there are a bunch of warnings and the like, and is also much slower.

@alexcrichton
Copy link
Member

Can you provide an example to look at? What's probably happening is that cargo is running rustc src/lib.rs and rustc --test src/lib.rs, and then after the lib finishes building it rust rustc tests/foo.rs -L target.

If you don't want to build your library with --test then you can specify test = false in the manifest.

@nixpulvis
Copy link
Author

Here is a run of cargo test on a project I'm working on. As you can see it's printing the errors twice. I would expect cargo test to only run rustc --test src/lib.rs && target/parser-<something>.

> cargo test
   Compiling parser v0.0.1 (file:///Users/nixpulvis/Code/parser)
/Users/nixpulvis/Code/parser/src/lexer.rs:15:7: 21:4 error: cannot infer an appropriate lifetime due to conflicting requirements
/Users/nixpulvis/Code/parser/src/lexer.rs:15:7: 21:4 error: cannot infer an appropriate lifetime due to conflicting requirements
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:18     } else {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
/Users/nixpulvis/Code/parser/src/lexer.rs:19       Error
/Users/nixpulvis/Code/parser/src/lexer.rs:18     } else {
/Users/nixpulvis/Code/parser/src/lexer.rs:20     }
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:19       Error
/Users/nixpulvis/Code/parser/src/lexer.rs:20     }
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:12:45: 22:2 note: first, the lifetime cannot outlive the block at 12:44...
/Users/nixpulvis/Code/parser/src/lexer.rs:12 pub fn lit<'a>(value: &'a str) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:13   let l = value.len();
/Users/nixpulvis/Code/parser/src/lexer.rs:14 
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:12:45: 22:2 note: first, the lifetime cannot outlive the block at 12:44...
/Users/nixpulvis/Code/parser/src/lexer.rs:12 pub fn lit<'a>(value: &'a str) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:13   let l = value.len();
/Users/nixpulvis/Code/parser/src/lexer.rs:14 
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:16:35: 16:40 note: ...so that captured variable `value` does not outlive the enclosing closure
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
                                                                               ^~~~~
/Users/nixpulvis/Code/parser/src/lexer.rs:16:35: 16:40 note: ...so that captured variable `value` does not outlive the enclosing closure
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
                                                                               ^~~~~
/Users/nixpulvis/Code/parser/src/lexer.rs:12:45: 22:2 note: but, the lifetime must be valid for the lifetime 'a as defined on the block at 12:44...
/Users/nixpulvis/Code/parser/src/lexer.rs:12 pub fn lit<'a>(value: &'a str) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:13   let l = value.len();
/Users/nixpulvis/Code/parser/src/lexer.rs:14 
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
/Users/nixpulvis/Code/parser/src/lexer.rs:12:45: 22:2 note: but, the lifetime must be valid for the lifetime 'a as defined on the block at 12:44...
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:12 pub fn lit<'a>(value: &'a str) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:13   let l = value.len();
/Users/nixpulvis/Code/parser/src/lexer.rs:14 
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
/Users/nixpulvis/Code/parser/src/lexer.rs:15:3: 21:4 note: ...so that it can be closed over into an object
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
/Users/nixpulvis/Code/parser/src/lexer.rs:18     } else {
/Users/nixpulvis/Code/parser/src/lexer.rs:19       Error
/Users/nixpulvis/Code/parser/src/lexer.rs:20     }
                                             ...
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:15:3: 21:4 note: ...so that it can be closed over into an object
/Users/nixpulvis/Code/parser/src/lexer.rs:15   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:16     if s.len() >= l && s[0..l] == value {
/Users/nixpulvis/Code/parser/src/lexer.rs:17       Success(box LiteralNode(s[0..l]), s[l..])
/Users/nixpulvis/Code/parser/src/lexer.rs:18     } else {
/Users/nixpulvis/Code/parser/src/lexer.rs:19       Error
/Users/nixpulvis/Code/parser/src/lexer.rs:20     }
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:25:7: 31:4 error: cannot infer an appropriate lifetime due to conflicting requirements
/Users/nixpulvis/Code/parser/src/lexer.rs:25:7: 31:4 error: cannot infer an appropriate lifetime due to conflicting requirements
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:30     }
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:30     }
                                             ...
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:24:59: 32:2 note: first, the lifetime cannot outlive the block at 24:58...
/Users/nixpulvis/Code/parser/src/lexer.rs:24:59: 32:2 note: first, the lifetime cannot outlive the block at 24:58...
/Users/nixpulvis/Code/parser/src/lexer.rs:24 pub fn alt<'a>(l1: Lexer<'a>, l2: Lexer<'a>) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:24 pub fn alt<'a>(l1: Lexer<'a>, l2: Lexer<'a>) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
                                             ...
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:29:25: 29:27 note: ...so that captured variable `l2` does not outlive the enclosing closure
/Users/nixpulvis/Code/parser/src/lexer.rs:29:25: 29:27 note: ...so that captured variable `l2` does not outlive the enclosing closure
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
                                                                     ^~
                                                                     ^~
/Users/nixpulvis/Code/parser/src/lexer.rs:24:59: 32:2 note: but, the lifetime must be valid for the lifetime 'a as defined on the block at 24:58...
/Users/nixpulvis/Code/parser/src/lexer.rs:24:59: 32:2 note: but, the lifetime must be valid for the lifetime 'a as defined on the block at 24:58...
/Users/nixpulvis/Code/parser/src/lexer.rs:24 pub fn alt<'a>(l1: Lexer<'a>, l2: Lexer<'a>) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:24 pub fn alt<'a>(l1: Lexer<'a>, l2: Lexer<'a>) -> Lexer<'a> {
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
                                             ...
                                             ...
/Users/nixpulvis/Code/parser/src/lexer.rs:25:3: 31:4 note: ...so that it can be closed over into an object
/Users/nixpulvis/Code/parser/src/lexer.rs:25:3: 31:4 note: ...so that it can be closed over into an object
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:25   box |&: s: &'a str| {
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:26     let r1 = (*l1).call((s,));
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:27     match r1 {
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:28       Success(_, _) => return r1,
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:29       Error => return (*l2).call((s,)),
/Users/nixpulvis/Code/parser/src/lexer.rs:30     }
/Users/nixpulvis/Code/parser/src/lexer.rs:30     }
                                             ...
                                             ...
error: aborting due to 2 previous errors
error: aborting due to 2 previous errors
Build failed, waiting for other jobs to finish...
Could not compile `parser`.

To learn more, run the command again with --verbose.

@alexcrichton
Copy link
Member

Do you have any binaries or integration tests ([test]) specified in the manifest or discovered implicitly? If I could take a peek at the code that would also be useful.

@nixpulvis
Copy link
Author

Here is the code. It's a tiny project at the moment because I can't figure out boxed unboxed closures 😜 but testing it with cargo test will yield duplicate warnings on master.

There are #[test]s.

@alexcrichton
Copy link
Member

Hm, did you mean to include a link in there?

@nixpulvis
Copy link
Author

nixpulvis commented Nov 13, 2014

Oh man, it's going to be a long night, already losing my mind. Here you go. https://github.com/nixpulvis/parser

@alexcrichton
Copy link
Member

Aha! That definitely shouldn't be compiling twice, thanks @nixpulvis! I'll take a look into this hopefully soon.

@nixpulvis
Copy link
Author

❤️

@SimonSapin
Copy link
Contributor

Isn’t the other compile a dependency of running rustdoc --test? You can disable this with doctest = false in Cargo.toml, but it would be nice if Cargo did that implicitly when it can figure out that the source does not contain any doctest. (Though I’m not sure how it would do that, maybe split rustdoc --test into doctest collection (which doesn’t require the crate to be built) and doctest compile/run?)

@nixpulvis
Copy link
Author

I don't think doctests should be opt in. I like that they are part of cargo test. It's just hard to read the warnings are errors as is.

This might be a bad idea, but maybe it actually might make sense to allow rustc to include doctests when passes an additional flag.

@SimonSapin
Copy link
Contributor

I don’t mean making doctest opt-in, only to avoid running zero doctests when Cargo can determine there are indeed zero, if that can be determine quickly.

@nixpulvis
Copy link
Author

But then the moment you add doctests then your back into the same problem.

@SimonSapin
Copy link
Contributor

I don’t see the problem. When you add doctests you don’t have zero doctests anymore and so Cargo should not detect that you have zero doctests anymore (again, assuming this detection is feasible) and would start running doctests again. This magic "detection" would run every time you run cargo test.

@nixpulvis
Copy link
Author

The problem is with the original problem of showing two compilations right next to each other with all their warnings and errors.

@SimonSapin
Copy link
Contributor

Ah, sorry I misunderstood. Yes indeed. I believe the reason for that is that rustdoc --test need a library to link against, but rustc --test creates an executable with the test harness, not a library.

@alexcrichton
Copy link
Member

Ah yes, @SimonSapin is right, I forgot about doc tests! For now I don't think Cargo will try to detect the presence or non-presence of doc tests as it would involve a dependency on rustdoc itself. If you'd like to just see one set of warnings at a time you can pass -j1 to ensure only one crate is built at a time. Otherwise this is currently working as intended.

@nixpulvis
Copy link
Author

This is going to be a pain point for adoption and usability.

Just my 2 cents.

@drewm1980
Copy link

I was experiencing this problem, or something similar. When I would run "cargo test" the tests defined in my library are getting run by the main executable as well as in the library, so they ran twice. Adding "test = false" for the library in my manifest fixed it.

Thanks Alex!

It doesn't matter now, but here is my code:
https://github.com/drewm1980/rust-omr

@jimmycuadra
Copy link
Contributor

I think this is a poor user experience. For quite a while I've been dealing with duplicate output from cargo test and assumed it was just a bug that was going to get fixed. Now that I've found this issue, I don't think it's at all obvious that the duplicate output is coming from attempting to run non-existent doctests. If there really isn't a way to disable running doctests when there are none, it'd be great if there were an obvious warning message (or some other form of easily discoverable documentation) that explains why the output shows up twice by default.

@SimonSapin
Copy link
Contributor

@jimmycuadra You can add doctest = false in the [lib] section of your Cargo.toml. (Though I still wish Cargo would do better by default.)

@jimmycuadra
Copy link
Contributor

Yes, that is what I did. It's just totally non-obvious that that's why you see double output by default.

@nixpulvis
Copy link
Author

This is still painful FWIW, getting a lot of unused warnings, and double errors on failed compiles.

@jimmycuadra
Copy link
Contributor

@alexcrichton I understand the technical reason why this happens now, but since it is still a confusing user experience (as noted by duplicates of this issue in various forms continuing to be opened), would you be amenable to either reopening this issue, or me opening a new issue specifically for discussion and tracking of possible ways to address user confusion about the output?

@alexcrichton
Copy link
Member

@jimmycuadra that's tracked by #1854

@lilith
Copy link
Contributor

lilith commented Nov 17, 2016

cargo test --lib fixes this for me, but not doctest = false under [lib]. Using nightly- 2016-11-04. Is this expected or should doctest=false always eliminate duplicate output?

(FWIW, I ignored duplicates for three months in hopes the "bug" would be fixed. If there's a pain points FAQ/wiki I'd be glad to contribute to it).

@alexcrichton
Copy link
Member

@nathanaeljones do you have integration tests (tests/*.rs) or binaries? In that case cargo test would also have to build the library for those targets.

@lilith
Copy link
Contributor

lilith commented Nov 17, 2016

Yes, I do. That makes sense, thanks!

@a13xb
Copy link

a13xb commented Dec 6, 2018

I'd echo the user experience complaints. As a newcomer, the current behaviour (still the case as of 1.30) is extremely baffling. I don't think it should be considered "fixed".

@Syndog
Copy link

Syndog commented Mar 5, 2020

I'm adding my voice to the chorus with this. Duplicate messages by default makes an extremely poor and unprofessional user experience. Though a minor issue, the fact that it continues to be pointedly ignored after so many years (now 1.41.0) only exacerbates matters. It reveals an administrative problem that doesn't speak well for Rust as a project.

@pauljherring
Copy link

Hello, from Jul 2020; 6 years later, and still a thing.

As with @lilith, [lib]doctest = false didn't work, cargo test --lib did.

@Iron-E
Copy link

Iron-E commented Dec 30, 2020

Latest nightly here. This makes it pretty difficult to find unique errors that I need to fix in the middle of a refactor:

cap

Especially when I don't just want to test just by --lib, I really do want to test everything.

@nixpulvis
Copy link
Author

@alexcrichton @SimonSapin I realize that this issue thread is kind of messy and apologize for that, but I do not feel like it should be considered closed. Would it be possible to reopen? or create a new issue without the mess for this bad user experience?

@ehuss
Copy link
Contributor

ehuss commented Jan 25, 2021

@nixpulvis I don't think this should be reopened because due to the way Rust testing works, the library fundamentally needs to be built twice. However, we have been planning to add some kind of deduplicate support. I realize we never opened an issue to track that, so I have opened #9104.

@nixpulvis
Copy link
Author

@ehuss awesome, thanks for doing that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests