Add rustc flags for cross-compiling only

[therealbstern]

I'm having the same problem as described in #3105, except that I'm not cross-compiling. (I'm opening a new issue because #4423 seems more focused upon cross-compiling, #3105 seems to be as well, and also #3105 was closed in favor of #4423).

Several crates that did not formerly need -C rpath for their custom build scripts now do, including html5ever, markup5ever, and string_cache.

I used the manual rust installation method, due to rust-lang/rustup#241 (I like having signatures of things checked before I install them.)

If I add -C rpath by hand to the command line for building the relevant build-script-build binaries (acquired from cargo build -vv), my subsequent cargo build works.

-C rpath isn't needed for the overall build: it certainly works if I put it in RUSTFLAGS but I don't need it for any target output other than the build-scripts-build binaries identified above. My final binaries don't require it, for example.

I'm on x86_64-unknown-linux-gnu and using:

• cargo 0.26.0 (41480f5cc 2018-02-26)
• rustc 1.25.0 (84203cac6 2018-03-25)

[matklad]

@therealbstern yeah, it is indeed a bug in Cargo there there's no way to set RUSTFLAGS separately for build scripts and for actual crates. However, I am a bit confused why you need to set RUSTFLAGS in the first place?

It seems to me that Cargo should be able to build all those crates without fiddling with rpath? You mention that you've installed rustup manually. Do you think that this is the reason why you need to set rpath explicitly? Cargo does not actually know about rustup and so it should work the same, regardless of the procedure which was used to install rust.

[therealbstern]

@matklad I don't use rustup at all, I installed rust using the tarball. This was suggested by others (out of band) as a cause for the problem. I am happy to believe that it is not. I too agree that the installation method should not be relevant.

The reason I need to set RUSTFLAGS is so that I can get the build-scripts-build to run. Specifically, adding -C rpath to the link line for build-scripts-build allows the OS dynamic loader to find the .so for proc_macro. (More information is at servo/html5ever#342, where I originally mentioned the problem, but was referred upstream. As above, this problem occurs with three different crates from Servo.)

According to @SimonSapin in the linked issue,

[html5ever's] build script (indirectly) uses the proc_macro crate, which apparently is only distributed as a .so file, no .rlib (unlike std for example which has both). I think this is related to the fact that procedural macros are usually loaded by the compiler through dynamic linking, but maybe the .rlib file should be distributed anyway.

[matklad]

Specifically, adding -C rpath to the link line for build-scripts-build allows the OS dynamic loader to find the .so for proc_macro.

I believe this step should not be necessary, because libprocmacro lives in compiler's sysroot, and compiler should make sure that the linker picks it up.

I've installed rust locally via tarball, and I am able to compile html5ever without setting LD_LIBRARY_PATH.

Could you check that your installation has sysroot setup correctly? You can use rustc --print sysroot to check this

Here's how it looks on my machine:

λ echo $RUSTC
/home/matklad/trash/rust/usr/local/bin/rustc

~/trash/html5ever master
λ ls $($RUSTC --print sysroot)/lib
.rw-r--r--  39k matklad 25 Apr 12:46 libarena-0ca7b9d43c18c972.so
.rw-r--r--  53k matklad 25 Apr 12:46 libfmt_macros-417151f3addcf248.so
.rw-r--r--  81k matklad 25 Apr 12:46 libgraphviz-fb561c6c2a5da190.so
.rw-r--r-- 320k matklad 25 Apr 12:46 libproc_macro-7d531857c463fdf1.so # <- this is proc_macro
.rw-r--r--  17M matklad 25 Apr 12:46 librustc-276311b1d0544da9.so
.rw-r--r-- 396k matklad 25 Apr 12:46 librustc_allocator-1b198b9b0310fa61.so
.rw-r--r-- 705k matklad 25 Apr 12:46 librustc_back-1798c92810b95c97.so
.rw-r--r-- 503k matklad 25 Apr 12:46 librustc_borrowck-fe33dc5b18b443df.so
.rw-r--r-- 497k matklad 25 Apr 12:46 librustc_const_eval-dc09479c804cb63d.so
.rw-r--r-- 431k matklad 25 Apr 12:46 librustc_const_math-90d5d06b1fe18189.so
.rw-r--r--  15k matklad 25 Apr 12:46 librustc_cratesio_shim-f858a3ee752a9bb9.so
.rw-r--r-- 888k matklad 25 Apr 12:46 librustc_data_structures-6c5726675bdb8f04.so
.rw-r--r-- 2.4M matklad 25 Apr 12:46 librustc_driver-3163d9fca1cf728b.so
.rw-r--r-- 455k matklad 25 Apr 12:46 librustc_errors-0442a147c1fbea3f.so
.rw-r--r-- 879k matklad 25 Apr 12:46 librustc_incremental-d3977159b7474393.so
.rw-r--r-- 647k matklad 25 Apr 12:46 librustc_lint-251ddd0c571f587a.so
.rw-r--r-- 2.2M matklad 25 Apr 12:46 librustc_metadata-8671c18130c60598.so
.rw-r--r-- 5.4M matklad 25 Apr 12:46 librustc_mir-8f021bbd0465cbb0.so
.rw-r--r-- 434k matklad 25 Apr 12:46 librustc_passes-6a76358a93a19970.so
.rw-r--r-- 1.1M matklad 25 Apr 12:46 librustc_platform_intrinsics-0d01b335c40e3988.so
.rw-r--r-- 136k matklad 25 Apr 12:46 librustc_plugin-23ad766384f35873.so
.rw-r--r-- 241k matklad 25 Apr 12:46 librustc_privacy-5f6879a02abcc654.so
.rw-r--r-- 1.3M matklad 25 Apr 12:46 librustc_resolve-aa7735a0f73cae37.so
.rw-r--r-- 1.3M matklad 25 Apr 12:46 librustc_save_analysis-d3fb0e87c4a6833f.so
.rw-r--r-- 478k matklad 25 Apr 12:46 librustc_trans_utils-bc5193ed188bba9c.so
.rw-r--r-- 2.9M matklad 25 Apr 12:46 librustc_typeck-7c1d49c584f4b735.so
.rw-r--r-- 774k matklad 25 Apr 12:46 libserialize-df58869bc5612287.so
.rw-r--r-- 8.4M matklad 25 Apr 12:46 libstd-c10c01f750e28d27.so
.rw-r--r-- 6.0M matklad 25 Apr 12:46 libsyntax-a0f8084e7d2e8f23.so
.rw-r--r-- 967k matklad 25 Apr 12:46 libsyntax_ext-673936b12cf0a541.so
.rw-r--r-- 380k matklad 25 Apr 12:46 libsyntax_pos-a3c77f19bf623ee8.so
.rw-r--r-- 659k matklad 25 Apr 12:46 libterm-74d3aea795746522.so
.rw-r--r-- 1.4M matklad 25 Apr 12:46 libtest-d35c28f0ef54928a.so
drwxr-xr-x    - matklad 25 Apr 12:46 rustlib

[matklad]

To give more details, I've installed rust by downloading the tarball from https://www.rust-lang.org/en-US/other-installers.html and running ./install.sh --destdir=~/trahs/rust, and this just worked.

[SimonSapin]

@therealbstern Could some other crate in your dependency graph be enabling the nightly feature of the proc-macro2 crate? I think that’s how proc_macro ends up being linked into html5ever’s build script.

[SimonSapin]

servo/html5ever#342 (comment)

This build script (indirectly) uses the proc_macro crate, which apparently is only distributed as a .so file, no .rlib (unlike std for example which has both). I think this is related to the fact that procedural macros are usually loaded by the compiler through dynamic linking, but maybe the .rlib file should be distributed anyway.

This is controlled by https://github.com/rust-lang/rust/blob/1.25.0/src/libproc_macro/Cargo.toml#L8

[therealbstern]

@matklad rustc --print sysroot shows /usr/local (which is correct, IMO).

@SimonSapin: I don't have nightly installed, so I would think that it would crash the build if I needed it. Is there a cargo command from which you would like to see the output?

[matklad]

@therealbstern and do you have /usr/local/lib/libproc_macro-7d531857c463fdf1.so file? I think you are supposed to have it (at least, for me the installation process copies rust libs directly to /usr/lib, and not only to rustlib/tripple/lib), and if you do, loader should be able to pick it up automatically.

[therealbstern]

@matklad I do. Here's what ldd is telling me on html5ever's build-script-build binary:

ldd target/debug/build/html5ever-8e14a6802a9b2fd9/build-script-build
        linux-vdso.so.1 (0x00007ffdc5188000)
        libproc_macro-7d531857c463fdf1.so => not found
        libstd-c10c01f750e28d27.so => not found
        libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x00007efea8dcd000)
        libc.so.6 => /lib64/libc.so.6 (0x00007efea8a04000)
        /lib64/ld-linux-x86-64.so.2 (0x00007efea9604000)

Following through...

$ ls /usr/local/lib/libproc_macro* /usr/local/lib/libstd-*
/usr/local/lib/libproc_macro-7d531857c463fdf1.so
/usr/local/lib/libstd-c10c01f750e28d27.so

$ ls  /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/lib{proc_macro,std}-*
/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libproc_macro-7d531857c463fdf1.so
/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-c10c01f750e28d27.rlib
/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-c10c01f750e28d27.so

$ arch
x86_64

As @SimonSapin pointed out, note the lack of libproc_macro-7d531857c463fdf1.rlib. Notwithstanding, libstd is in my library search path, and ldd can't find it either.

However, this detective work was of value, at least to me:

$ cat /etc/ld.so.conf
/lib64
/usr/lib64
/usr/local/lib64
/usr/x86_64-slackware-linux/lib64

I apologize for wasting time on a search path problem. I do posit that there is an installer problem, since on x86_64, lib64 is arguably more correct, but it's not cargo's fault.

As mentioned in #5411 (comment), do you want an issue on cargo not allowing RUSTFLAGS for build scripts, or should I let sleeping dogs lie? ;-)

[matklad]

Let's close this!

Although the underlying issue in #4423 is different, the desirable feature of being able to specify separate HOST_RUSTFLAGS and TARGET_RUSTFLAGS is the same!

I do posit that there is an installer problem, since on x86_64, lib64 is arguably more correct, but it's not cargo's fault.

Probably! I would suggest searching for an existing issue at rust-lang/rust, and opening a new one if this wasn't reported before!