deduplicate the native and cross bootstraps

[ghost]

There is a huge amount of duplication between the cross bootstrap and stdenv/linux. Both of them go through the same dance. The goal is to deduplicate this dance with no loss of functionality.

Here is what the cross bootstrap looks like:

0. Build pkgsBuildBuild.stdenv
1. Use pkgsBuildBuild.stdenv to compile the bintools (bintoolsNoLibc)
2. Use pkgsBuildBuild.stdenv to compile a bare-bones compiler (gccWithoutTargetLibc / clangNoLibc)
3. Use the bare-bones compiler to compile the bintools
4. Use the bare-bones compiler to compile the libc
5. Use the bare-bones compiler to compile real compiler
6. Use the real compiler to compile the libc

Here are the native linux stages:

0. Unpack the bootstrap-files
1. Use the bootstrap-files compiler+bintools to compile the bintools
2. Use the bootstrap-files compiler and new bintools to the bare-bones compiler (xgcc)
3. Use the bare-bones compiler to compile the bintools
4. Use the bare-bones compiler to compile the libc
5. Use the bare-bones compiler to compile the real compiler
6. Use the real compiler to compile the libc
7. Use the real compiler to compile the bintools

The only native bootstrap step that doesn't correspond to anything in the cross bootstrap is the very last one. It exists only because the native bootstrap has a disallowedReferences entry for the bootstrap-files, but cross bootstrap doesn't care about that. This stage would simply be skipped in the cross case. Or we could keep it and add the disallowedReferences check to the cross bootstrap as well.

Ultimately what will happen is that instead of taking a bootstrap-files, stdenv/linux/default.nix as an argument, it will instead take any stdenv as an argument. That argument can then be any of these:

• an unpacked bootstrap-files (which has the same approximate shape as stdenv)
• pkgsBuildBuild.stdenv
• pkgsCross.${targetPlatform}.freshBootstrapFiles, providing a simple and straightforward way to say "rebuild the bootstrap-files and then use them to bootstrap" (useful for testing)
• the minimal-bootstrap end product
• pkgsCross.${firstPlatform}.pkgsCross.${secondPlatform}.pkgsBuildTarget.stdenv, allowing to chain together cross compilations
  • this is useful with the minimal-bootstrap, since that only supports 32-bit x86.

[reckenrode]

I’m not suggesting to add it to the scope of this issue, but I’m watching it with interest from the Darwin side to see if there’s anything that can be brought over. I have plans for Darwin cross in 24.05, and one of the major pains with fixing it is how the stdenv and cross bootstraps are so different.