Miri (possibly everything that depends on rustc private crates) no longer builds: cannot find -lLLVM-18-rust-1.78.0-nightly #121889
Comments
rustbot
added
the
needs-triage
|
How does miri link against rust? |
|
Just a bunch of extern crate rustc_apfloat;
extern crate rustc_ast;
extern crate rustc_const_eval;
extern crate rustc_data_structures;
extern crate rustc_errors;
extern crate rustc_hir;
extern crate rustc_index;
#[macro_use]
extern crate rustc_middle;
extern crate rustc_session;
extern crate rustc_span;
extern crate rustc_target;
// Necessary to pull in object code as the rest of the rustc crates are shipped only as rmeta
// files.
#[allow(unused_extern_crates)]
extern crate rustc_driver; |
|
Given that rustc CI still works and only builds against the distributed rustc toolchain are affected, it seems likely that the issue is related to the rustc-dev component -- maybe that doesn't contain all the required files any more? |
Noratrieb
added
regression-from-stable-to-nightly
|
Reproducing instructions:
|
The necessary symlink is indeed not part of rustc-dev (only rust-dev). We could add it, but it's not obvious to me why it is needed. I'm not familiar with how rustc generates that linker invocation, but the fact that it explicitly includes |
There is no
I don't know anything about this linking magic either.^^ Cc @bjorn3 |
|
But evidently the symlink is needed... so in the mean time it'd be good to unstuck this by adding the symlink I think. I expect this will break out-of-tree builds of Clippy as well, and rustfmt -- basically every tool that links against rustc. |
Ah, it's not a real component, just the tarball that download-ci-llvm uses.
Main problem is that we have checks against shipping symlinks, and I expect they exist for good reason... |
|
Ah, yeah probably Windows doesn't like them. Why can't rustc_driver link against the correct .so file, i.e., why is a symlink needed in the first place? |
|
I think this is the reason why it gets added to the linker line: rust/compiler/rustc_codegen_ssa/src/back/link.rs Lines 2708 to 2711 in 4cdd205
This is because llvm-config returns something like Possibly that's a bug? Similar to how shared objects embed the resolved name, maybe rustc should do the same for its own metadata. Though independent of general rustc behavior, I guess we could explicitly resolve the symlink in the build.rs for rustc_llvm. That might be the most straightforward fix. |
If llvm-config inherently relies on a symlink, how do they support Windows? |
That would break compiling an rlib which states that it links against a certain dylib when said dylib is not available on the host, right? Currently you only need dylibs to be available when actually linking (that is using a crate type other than rlib or staticlib).
It includes By the way |
|
I just looked at the latest nightly and |
|
I got confused about when #121395 got merged. I though it was already included in the latest nightly. I just looked at the llvm-preview component for that PR and it includes |
|
Why can't rustc_driver include |
|
Because the linker will prefix the name with |
Windows doesn't have a concept of sonames, so I don't think a symlink would be used there. I don't know what the dylib is called there. We use static linking there anyway. So I think we wouldn't actually ship a symlink on anything but Linux. More problematic is the other part of this comment: rust/src/bootstrap/src/utils/tarball.rs Lines 346 to 347 in 1090205 If rustup-toolchain-install-master can't deal with symlinks, that's also a problem for Linux. |
Any platform except Windows you mean? |
As far as I can tell, dist-x86_64-linux is the only configuration where we link LLVM dynamically. (Well and the llvm-16/llvm-17 images, but that's non-dist and a different situation anyway.) |
|
So to summarize, I see a few options here:
I think 3 is probably ideal long term, but depends on just how broken symlink support is right now. Is this a matter of "we'll copy the file instead of symlinking" or "rustup-toolchain-install-master is going to abort"? Maybe @Mark-Simulacrum knows. The easiest fix would be 1. |
|
There's another fix, which is to ship a |
|
Oooh, that's a great idea! |
|
I got klint CI working with the fix: Rust-for-Linux/klint@277cb26 |
|
Option 6, convince LLVM that this way of naming libraries is a bad idea and causes problems? I can't tell to what extent this is caused by Rust idiosyncrasies vs LLVM doing something strange.
Wait, a .so file can be just a text file...?!? |
Basically, LLVM used to do it's own thing here, and now follows the standard convention for shared object naming on Linux. This turns out to be quite inconvenient for us, but I don't think it would be reasonable to ask LLVM to undo the change. (Though, imho, this change really should not have been done in an rc3 release...)
It can be a linker script. Somewhat unusual, but things like libc.so or libcxx.so are often linker scripts. |
|
Well, if that linker script solution works that would be great. :) |
Replace libLLVM symlink with linker script It turns out that the libLLVM-N.so -> libLLVM.so.N.1 symlink is also needed when projects like miri link against librustc_driver.so. As such, we have to distribute it in real rustup components like rustc-dev, rather than only for download-ci-llvm. To avoid actually distributing symlinks (which are not supported or not fully supported by the rustup infrastructure) replace it with a linker script that does the same thing instead. Fixes rust-lang#121889. r? `@cuviper`
Replace libLLVM symlink with linker script It turns out that the libLLVM-N.so -> libLLVM.so.N.1 symlink is also needed when projects like miri link against librustc_driver.so. As such, we have to distribute it in real rustup components like rustc-dev, rather than only for download-ci-llvm. To avoid actually distributing symlinks (which are not supported or not fully supported by the rustup infrastructure) replace it with a linker script that does the same thing instead. Fixes rust-lang#121889. r? `@cuviper`
|
This is now beginning to block Miri development. If someone reading along knows enough about linkers to approve #121967 that would be great. :) |
Replace libLLVM symlink with linker script It turns out that the libLLVM-N.so -> libLLVM.so.N.1 symlink is also needed when projects like miri link against librustc_driver.so. As such, we have to distribute it in real rustup components like rustc-dev, rather than only for download-ci-llvm. To avoid actually distributing symlinks (which are not supported or not fully supported by the rustup infrastructure) replace it with a linker script that does the same thing instead. Fixes rust-lang/rust#121889. r? `@cuviper`
Trying to build Miri against latest rustc artifacts fails:
Regression range: 6f435eb...eaee1e9
Likely cause: #121395
Cc @nikic
The text was updated successfully, but these errors were encountered: