Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustbuild: Tweak LLVM distribution layout #53955

Merged
merged 1 commit into from
Sep 6, 2018

Conversation

alexcrichton
Copy link
Member

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at $sysroot/lib. The LLVM tools we ship are
in two locations:

  • LLD is shipped at $sysroot/lib/rustlib/$host/bin/rust-lld
  • Other LLVM tools are shipped at $sysroot/bin

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to ../lib and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at $sysroot/bin, not LLD!

This commit is targeted at fixing this situation by making two changes:

  • LLVM tools other than LLD are moved in the distribution to
    $sysroot/lib/rustlib/$host/bin. This moves them next to LLD and should
    position them for...
  • The LLVM shared object is moved to $sysroot/lib/rustlib/$host/lib

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes #53813

@rust-highfive
Copy link
Collaborator

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 4, 2018
@alexcrichton
Copy link
Member Author

cc @japaric, this is the change I mentioned on IRC

r? @kennytm

@rust-highfive rust-highfive assigned kennytm and unassigned aturon Sep 4, 2018
Copy link
Member

@japaric japaric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. (You can r=me if that makes sense)

@cuviper
Copy link
Member

cuviper commented Sep 4, 2018

cc #53852 @Keruspe - this may solve your issue by moving libLLVM under rustlib/.

@alexcrichton
Copy link
Member Author

@bors: try

Gonna run this through bors to double check it's working as expected.

@bors
Copy link
Contributor

bors commented Sep 4, 2018

⌛ Trying commit ace1929 with merge b07b6be...

bors added a commit that referenced this pull request Sep 4, 2018
rustbuild: Tweak LLVM distribution layout

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:

* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!

This commit is targeted at fixing this situation by making two changes:

* LLVM tools other than LLD are moved in the distribution to
  `$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
  position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes #53813
@bors
Copy link
Contributor

bors commented Sep 4, 2018

☀️ Test successful - status-travis
State: approved= try=True

@Keruspe
Copy link
Contributor

Keruspe commented Sep 5, 2018

This would partly fix #53852 too, but stable, beta and nightly would install the same file (once it reaches stable), so adding a way to customize LLVM_VERSION_SUFFIX will still be needed

@alexcrichton
Copy link
Member Author

@bors: r=japaric

@bors
Copy link
Contributor

bors commented Sep 5, 2018

📌 Commit 60584a36dbac1427f341ab073c1e9cce0e89d869 has been approved by japaric

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 5, 2018
@alexcrichton
Copy link
Member Author

@bors: p=1

(fixing a regression for a number of targets on nightly)

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In rust-lang#53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:

* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!

This commit is targeted at fixing this situation by making two changes:

* LLVM tools other than LLD are moved in the distribution to
  `$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
  position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes rust-lang#53813
@alexcrichton
Copy link
Member Author

@bors: r=japaric

@cuviper
Copy link
Member

cuviper commented Sep 5, 2018

@Keruspe if you're trying to install multiple versions to a shared sysroot, how do you deal with the existing conflicts? At least bin/ and lib/rustlib/$target/codegen-backends/ are not uniquely named.

@Keruspe
Copy link
Contributor

Keruspe commented Sep 5, 2018 via email

@alexcrichton
Copy link
Member Author

@bors: r=japaric

@bors
Copy link
Contributor

bors commented Sep 5, 2018

📌 Commit bce09b6 has been approved by japaric

@bors
Copy link
Contributor

bors commented Sep 6, 2018

⌛ Testing commit bce09b6 with merge 27e5457...

bors added a commit that referenced this pull request Sep 6, 2018
rustbuild: Tweak LLVM distribution layout

This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.

LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:

* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`

Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!

This commit is targeted at fixing this situation by making two changes:

* LLVM tools other than LLD are moved in the distribution to
  `$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
  position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`

Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...

Closes #53813
@bors
Copy link
Contributor

bors commented Sep 6, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: japaric
Pushing 27e5457 to master...

@bors bors merged commit bce09b6 into rust-lang:master Sep 6, 2018
tromey added a commit to tromey/rust that referenced this pull request Sep 7, 2018
We're shipping a rust-enabled lldb, but the "lldb" executable is not
installed into the "bin" directory by rustup.  See the discussion in
rust-lang/rustup#1492 for
background on this decision.  There, we agreed to have rust-lldb
prefer the rust-enabled lldb if it is installed.

This patch changes dist.rs to put lldb into rustlib, following what
was done for the other LLVM tools in rust-lang#53955, and then fixes rust-lldb
to prefer that lldb, if it exists.

See issue rust-lang#48168
@alexcrichton alexcrichton deleted the fix-dist-again branch September 27, 2018 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rust-lld fails with "dyld: Library not loaded: @rpath/libLLVM.dylib" on 1.30.0-nightly (02cb8f2a4 2018-08-29)
8 participants