stdenv: add pure Linux stdenv support for powerpc64le and sparc64 #141448

piperswe · 2021-10-13T03:38:43Z

Motivation for this change

I have a sparc64 system I want to try to get NixOS running on, and if I can get NixOS working well in a powerpc64le emulator, I'll consider getting a ppc64le server to play around with (and probably use as a Hydra buildbox so I can add ppc64le to my binary cache).

I've been building things for this branch on my Hydra instance as part of my nixfiles project/jobset: https://hydra.piperswe.me/jobset/nixfiles/main

The results of basically all of my builds go into my binary cache, so if you want to play around with this branch you can use my binary cache to speed things up:

{
  nix = {
    binaryCaches = [
      "https://nix-cache.piperswe.me"
    ];
    binaryCachePublicKeys = [
      "nix-cache.piperswe.me:4r7vyJJ/0riN8ILB+YhSCnYeynvxOeZXNsPNV4Fn8mE="
    ];
  };
}

Since changing stdenv requires rebuilding the world, I might also add a few more interesting architectures to this PR so that we have bootstrap tools for most architectures out there at the cost of one world rebuild.

Fixes #113977. Depends on #141451.

Things done

r-burns · 2021-10-13T03:55:56Z

Cool, I'm glad there are others interested in exotic platforms. Feel free to ping me for review on any related PRs.

W.r.t. bootstrap tools, see also #115406. These always get held up because it's hard to verify that the bootstrap tools are reproducible - how do you do that for native bootstrap tools on a platform we don't support yet? But on the other hand, cross-compiled bootstrap tools won't build stdenv because of #113977, so I'm not really sure how to proceed here.

Your first commit (b8eebed) looks pretty noncontroversial, if you'd like to split that off into a separate PR we could probably get that merged quickly and start playing around with cross-compiled sparc64 packages.

piperswe · 2021-10-13T04:44:55Z

W.r.t. bootstrap tools, see also #115406. These always get held up because it's hard to verify that the bootstrap tools are reproducible - how do you do that for native bootstrap tools on a platform we don't support yet? But on the other hand, cross-compiled bootstrap tools won't build stdenv because of #113977, so I'm not really sure how to proceed here.

Very true... it seems like there are three stages to building bootstrap tools:

Cross-compile
Native using cross tools
Native using native tools

Maybe there should be more automation around building these - like, on a system with qemu-binfmt, you have a derivation that runs through all three stages, though that requires having a way to inject the newly built bootstrap tools into nixpkgs.

The first approach that comes to mind would be adding an argument to nixpkgsFun for the seeds, which probably wouldn't be a bad idea in general. I'd love to hear other people's thoughts on that though.

piperswe · 2021-10-13T20:36:44Z

pkgs/stdenv/linux/default.nix

+        stdenv = self.makeStaticLibraries self.stdenv;
+      }).overrideAttrs (oa: {
+        # Work around faulty stackprotector support in cross bootstrap tools
+        hardeningDisable = (oa.hardeningDisable or []) ++ [ "stackprotector" ];


See #113977 / #113977 (comment)

pkgs/os-specific/linux/musl/default.nix

I have no clue if this is the right value, but it seems like it probably is

SuperSandro2000 · 2021-11-15T10:39:36Z

pkgs/os-specific/linux/musl/default.nix

+    platforms   = [
+      "i386-linux" "x86_64-linux"
+      "armv5tel-linux" "armv6l-linux" "armv7l-linux" "armv7a-linux" "aarch64-linux"
+      "mipsel-linux"
+      "powerpc64-linux" "powerpc64le-linux"
+      "or1k-linux"
+      "s390x-linux"
+      "riscv64-linux"
+    ];


This is basically platforms.linux? Or which platform is not supported? I can't tell. It would be way cleaner to substract the one or two not working platforms from platforms.linux or just write them into meta.broken.

SuperSandro2000 · 2021-11-15T10:40:57Z

pkgs/stdenv/cross/default.nix

@@ -1,5 +1,5 @@
 { lib
-, localSystem, crossSystem, config, overlays, crossOverlays ? []
+, localSystem, crossSystem, config, overlays, crossOverlays ? [], ...


Why do we suddenly need to accept any input here and in the other files below? I don't think that is a good idea because it swallows mistakes like typos and makes.

SuperSandro2000 · 2021-11-15T10:44:23Z

pkgs/stdenv/linux/scratch/stage1.nix

@@ -0,0 +1,18 @@
+# Stage 1 is to cross-compile seeds.


This and the 3 files below are very similar. Could we combine them?

SuperSandro2000 · 2021-11-15T10:44:59Z

pkgs/stdenv/linux/scratch/default.nix

@@ -0,0 +1,9 @@
+# Usage: nix-build ./default.nix --argstr to powerpc64le-linux -A dist
+{ from ? builtins.currentSystem, to, nixpkgs ? import ../../../.. }:


That could break with flakes because currentSystem is not defined there.

SuperSandro2000 · 2021-11-15T10:46:03Z

pkgs/stdenv/linux/make-bootstrap-tools.nix

+    if crossSystem == null
+    then "-${localSystem.system}"
+    else "-${localSystem.system}-to-${crossSystem.system}";
+  stageSuffix = if stage == null then "" else "-${stage}";


Suggested change

stageSuffix = if stage == null then "" else "-${stage}";

stageSuffix = lib.optionalString (stage != null) "-${stage}";

SuperSandro2000 · 2021-11-15T10:46:38Z

pkgs/stdenv/linux/make-bootstrap-tools.nix

+  systemSuffix =
+    if crossSystem == null
+    then "-${localSystem.system}"
+    else "-${localSystem.system}-to-${crossSystem.system}";


Suggested change

systemSuffix =

if crossSystem == null

then "-${localSystem.system}"

else "-${localSystem.system}-to-${crossSystem.system}";

systemSuffix = "-${localSystem.system}" + lib.optionalString (crossSystem != null) "-to-${crossSystem.system}";

github-actions bot added the 6.topic: stdenv Standard environment label Oct 13, 2021

r-burns added the 6.topic: exotic Exotic hardware or software platform label Oct 13, 2021

piperswe added 2 commits October 12, 2021 23:00

stdenv: start working on adding more bootstrap tools

5bf36e5

lib/tests: add sparc64-linux to systems.nix

6957943

piperswe mentioned this pull request Oct 13, 2021

stdenv: add sparc64 cross-compilation support #141451

Open

12 tasks

piperswe mentioned this pull request Oct 13, 2021

stdenv: move seeds to a central place and allow overriding #141458

Closed

12 tasks

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Oct 13, 2021

piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from d45e137 to aba619a Compare October 13, 2021 07:21

piperswe commented Oct 13, 2021

View reviewed changes

r-burns reviewed Oct 13, 2021

View reviewed changes

pkgs/os-specific/linux/musl/default.nix Outdated Show resolved Hide resolved

piperswe added 2 commits October 13, 2021 18:57

lib: flesh out doubles

0115834

lib: clean up sparc64

b032580

piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from 58f75c4 to c1997a4 Compare October 14, 2021 00:00

piperswe added 2 commits October 13, 2021 19:34

lib: flesh out tests/systems

023ce07

busybox: don't use musl on risc-v and sparc

6171d9e

piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from 020ac9e to 9468c68 Compare October 14, 2021 01:02

busybox/sandbox-shell: fix eval on sparc64

b411f7e

piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from 9468c68 to ad6841c Compare October 14, 2021 01:41

piperswe added 6 commits October 13, 2021 23:13

stdenv: re-add useMusl to busyboxMinimal

0c54199

pcre: disable JIT on SPARC

1e31925

stdenv: add bootstrap tools for ppc64le and sparc64

1a06f19

stdenv: add derivation for building seeds from scratch

1e4c597

release: build bootstrap-tools from scratch in hydra

90916b3

lib: make qemuSerialDevice compatible with SPARC

dbe1471

I have no clue if this is the right value, but it seems like it probably is

piperswe added 9 commits October 13, 2021 23:34

release: attempt to fix eval

b5d08be

stdenv: fix building stdenv with cross-bootstrap

0876acb

musl: fail in eval for unsupported platforms

6a4def6

mips: fix powerpc eval

a2ecc8f

musl: limit to Linux

d80a804

musl: fix predicates

66c8fb2

musl: move platforms into doubles for use elsewhere

9d75aeb

musl: remove supportsMusl

50a957a

stdenv: add new stage building hardened gcc

1e268db

piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from ad6841c to 1e268db Compare October 14, 2021 04:34

stdenv: fix multi-stage bootstrap tools build

c303c89

github-actions bot added the 6.topic: bsd Running or building packages on BSD label Oct 14, 2021

piperswe added 4 commits October 14, 2021 12:40

stdenv: pass bootstrapFiles more elegantly

3e84ebc

stdenv: add pkgsWithTools to make-bootstrap-tools

c550f2f

stdenv: import pkgs.path instead of nixpkgs

06a046f

stdenv: fix eval

44cdd2c

r-burns mentioned this pull request Oct 23, 2021

stdenv: bootstrap riscv64-linux (Step 1) #115406

Merged

SuperSandro2000 reviewed Nov 15, 2021

View reviewed changes

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 19, 2022

ghost mentioned this pull request Jul 17, 2022

stdenv: lib{gmp,mpc,mpfr,isl}-stage3: isPower64 -> no -fstack-protector #181802

Closed

3 tasks

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 20, 2024

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stdenv: add pure Linux stdenv support for powerpc64le and sparc64 #141448

stdenv: add pure Linux stdenv support for powerpc64le and sparc64 #141448

piperswe commented Oct 13, 2021 •

edited

Loading

r-burns commented Oct 13, 2021

piperswe commented Oct 13, 2021 •

edited

Loading

piperswe Oct 13, 2021

SuperSandro2000 Nov 15, 2021

SuperSandro2000 Nov 15, 2021

SuperSandro2000 Nov 15, 2021

SuperSandro2000 Nov 15, 2021

SuperSandro2000 Nov 15, 2021

SuperSandro2000 Nov 15, 2021

		@@ -0,0 +1,9 @@
		# Usage: nix-build ./default.nix --argstr to powerpc64le-linux -A dist
		{ from ? builtins.currentSystem, to, nixpkgs ? import ../../../.. }:

	stageSuffix = if stage == null then "" else "-${stage}";
	stageSuffix = lib.optionalString (stage != null) "-${stage}";

stdenv: add pure Linux stdenv support for powerpc64le and sparc64 #141448

Are you sure you want to change the base?

stdenv: add pure Linux stdenv support for powerpc64le and sparc64 #141448

Conversation

piperswe commented Oct 13, 2021 • edited Loading

Motivation for this change

Things done

r-burns commented Oct 13, 2021

piperswe commented Oct 13, 2021 • edited Loading

piperswe Oct 13, 2021

Choose a reason for hiding this comment

SuperSandro2000 Nov 15, 2021

Choose a reason for hiding this comment

SuperSandro2000 Nov 15, 2021

Choose a reason for hiding this comment

SuperSandro2000 Nov 15, 2021

Choose a reason for hiding this comment

SuperSandro2000 Nov 15, 2021

Choose a reason for hiding this comment

SuperSandro2000 Nov 15, 2021

Choose a reason for hiding this comment

SuperSandro2000 Nov 15, 2021

Choose a reason for hiding this comment

piperswe commented Oct 13, 2021 •

edited

Loading

piperswe commented Oct 13, 2021 •

edited

Loading