Testing a cross-compiled toolchain

[jfgoog]

We (Android platform) build a custom version of rustc. Basically, we take the source distro, apply a few patches, and run x.py with a custom config.toml.

I am working on 2 things, which are related:

• We want to build a Rust toolchain that runs on Windows (MinGW). For various reasons, we'd prefer to do this by cross-compiling (i.e. build a Windows rustc on a Linux machine).

• We want to test our toolchain in our CI pipeline. For example, run x.py test tests/ui

So, I had the following "clever" idea:

1. Cross-compile a windows toolchain on Linux.
2. Copy it to a Windows machine.
3. In Windows, x.py --stage 0 test tests/ui with the stage 0 / bootstrap configured to use my new toolchain.

I got a very helpful warning that I probably shouldn't be doing this, and inviting me to file this issue. When I proceeded anyway, with COMPILETEST_FORCE_STAGE0=1, I got a number of test failures (~200 out of ~13,000). (For comparison, when I ran x.py test tests/ui the "right" way, the tests all passed, so I'm pretty confident the failures weren't due to our patches and custom configuration.)

Is there a correct, supported way to do this? That is, run tests "after-the-fact" on a toolchain you just built, possibly on a different machine? If not, I am willing to add support for it, if there is interest.

I found several things that seemed related, but not quite right:

• According to the docs, ./x.py test distcheck applies to a source tarball, not a compiled toolchain.
• The remote test server seems to be for testing different targets, not for when the toolchain itself is cross-compiled. It also seems like it would be very difficult to use in our CI pipeline. It also seems like it poses security issues in a production environment (opening a port allowing arbitrary code execution).

[jfgoog]

I took a look at some of the test failures, and they all seem to be related to the verbosity of error messages. For example, src/test/ui/borrowck/borrowck-move-out-of-overloaded-auto-deref.rs reports the following diff:

9          |
10      note: this function takes ownership of the receiver `self`, which moves value
11        --> $SRC_DIR/core/src/iter/traits/collect.rs:LL:COL
-          |
-       LL |     fn into_iter(self) -> Self::IntoIter;
-          |                  ^^^^
15
16      error: aborting due to previous error
17

From digging into the code, I wonder if -Zui-testing should be forcing the extra context to be printed, but sometimes does not.

[the8472]

Maybe copy over the output toolchain into the appropriate build/<target>/stage1* directories and then run tests with --keep-stage? Low confidence in that it'll work since I haven't tried that.

[jfgoog]

I tried your suggestion, and it doesn't work. Or, at least, it's not straightforward. The contents of the stageX directories don't neatly correspond to a packaged toolchain. For example, I have stage1, stage1-std, and stage1-rustc directories, so I would probably have to do some surgery on the packaged toolchain and figure out where each bit came from, and where to restore it. It seems like that is likely to be brittle.

It also did a bunch of unnecessary work, including building LLVM, before finally failing with:

Copying stage0 rustc from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Assembling stage1 compiler (x86_64-unknown-linux-gnu)
Warning: Using a potentially old libstd. This may not behave well.
Copying stage1 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
thread 'main' panicked at 'fs::read(stamp) failed with No such file or directory (os error 2) ("/usr/local/google/home/jamesfarrell/src/rust-toolchain/out/rustc/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/.libstd.stamp")', lib.rs:1421:24

[jyn514]

I took a look at some of the test failures, and they all seem to be related to the verbosity of error messages. For example, src/test/ui/borrowck/borrowck-move-out-of-overloaded-auto-deref.rs reports the following diff:

This is about missing the source for libstd, not the verbosity. Don't know why that would vary between machines; maybe the absolute path to the repository is getting embedded into rustc at build time somehow?

In general, I don't want to spend very much time at all trying to support this; bootstrap already tries to support far too many use cases and we have no way to test "a toolchain built on one machine but run on another" in CI.

[jfgoog]

This is helpful. I will see if I can figure out why we can't find the libstd sources.

I sympathize with your observation about testability. It would improve our testing, but make it harder for you.

[jfgoog]

Just to close the loop, since I didn't mention it earlier: It is possible for us to build natively on Windows, just more awkward. We will probably try that route, which has the advantage of being what the official version of x86_64-pc-windows-gnu does.