Tests fail with conflicting external dependencies in the target directory

[camsteffen]

This is about the "multiple matching crates found" error or "multiple rlibs found" error that occurs rather frequently when running Clippy UI tests.

There are already multiple issues opened, but I want to explain the problem, tell what has been done to try and fix it, centralize discussion and possibly find a solution.

In short, the current state of things is okay/working, but fragile. For instance, every time I run cargo check during local development, or build with a different set of feature flags, I bump into this error and have to rm target/debug/deps/lib* and try again. The error also occurs now and then in CI and it's a real show stopper.

One thing to keep in mind is that this error can be seen as a good thing as it may prevent rustc bootstrapping from needlessly recompiling external crates, adding more time to a process that we want to quicken in any way we can. So a fix along the lines of "just rebuild in another target directory" may not be desirable.

Detailed Explanation

Clippy UI tests use external crate dependencies from crates.io. It is the only project in the rustc repository with this feature. For instance, we have some lints that are specific to serde. We use (a fork of) compiletest like rustc. And compiletest does not have first-class support of this feature. In order to use external dependencies with compiletest, you need to have them compiled beforehand (by cargo) and provide -L dir or --extern file rust flags to compiletest. This is what Clippy does. We provide the Clippy target directory (e.g. target/debug/deps) so that any crates listed in the dependencies of the main Cargo.toml will be compiled and available for use in UI tests with extern crate.

This fails when there are multiple versions of the same library in the -L target dir(s) because rustc won't know which one to use and will fail with a "multiple matching crates" error. As long as only one version of every library is compiled, with the same features, everything is fine. That is basically the current solution. But a variety of scenarios can break this requirement. It's hard to maintain.

We have one partial fix where we give preference to libs in the clippy (or stage-tools) target dir by scanning the directory and using --extern with libs found there. This is where we have our dreaded "multiple rlibs found" panic case.

Of course, Cargo knows how to handle (and create) the "multiple rlibs" problem. So UI tests should just use Cargo? Well FWIW that's what trybuild does. But I don't think we want to add a third-party solution to the mix when we already have compiletest.

One potential solution is to run cargo build something --message-format=json and parse the output to get specific rlib paths and then use those paths as --extern flags with compiletest. This was previously done in #5121 but then reverted (#5145) when complications arose with rustc bootstrapping. I think that attempt was on the right track. The main problem though, I think, is that rustc bootstrapping has a very complex setup for running cargo commands. If clippy tests themselves run cargo commands during rustc bootstrapping but outside of the rustc bootstrapping framework, we're bound to do it wrong.

Goal

We should have a way to build UI test dependencies in the same target directory as Clippy itself and be able to use those dependencies without any risk of "multiple crates found". I think we should leverage cargo's dependency resolution abilities in some way. And it has to work within rustc bootstrapping.

Questions

• Should compiletest itself run cargo or resolve external dependencies more intelligently somehow?
• Can we use cargo build --message-format within Clippy tests (as in Improve compile_test and dogfood tests #5121) and make it work in rustc bootstrapping with the right set of env vars? (see FIXME)
• Should rustc bootstrap run cargo build --message-format instead and somehow provide the dependency info to Clippy tests?
• Maybe instead of having UI test dependencies in the main Cargo.toml, we should have another crate for UI tests and the Cargo.toml for that crate should have UI test dependencies declared instead. I'm not sure if that solves anything, but it's an idea that can be integrated with other solutions.

---

Related: #6809, #3643, rust-lang/rust#76495, rust-lang/rust#78717

[jyn514]

One thing to keep in mind is that this error can be seen as a good thing as it may prevent rustc bootstrapping from needlessly recompiling external crates, adding more time to a process that we want to quicken in any way we can. So a fix along the lines of "just rebuild in another target directory" may not be desirable.

Again, I strongly encourage you to measure the performance difference here before ruling it out. It is both the simplest and easiest fix.

[camsteffen]

I don't see that as a fleshed out solution that could be tested or ruled out.

[fpoli]

I don't know how much effort is needed to run cargo with rustc bootstrapping, but since you mentioned trybuild, an alternative might be the cargo-test-support library that is used by cargo's test suite. I wonder if they also need to run their tests with rustc bootstrapping. If, so there might already be a solution.