GHC on ARM platforms

[dhess]

Issue description

GHC 8.2.2 does not bootstrap on either armv7l-linux or aarch64-linux.

(However, ghc802 is currently working on armv7l-linux in as of aecc868)

Technical details

Now that #29688 is merged, it should be possible to build GHC and haskellPackages on both ARM platforms. I have an an overlay that I've been using for GHC on ARM for several months, so it definitely works with a bit of hand-holding for ARM.

I propose we track the remaining impediments here:

• Currently ghc822 bootstraps using ghc7103Binary. That will never work for aarch64-linux so we should bootstrap from ghc821Binary, instead. I have opened ghc: bootstrap 8.2.2 with 8.2.1-binary. #33350 for this change because it should probably be done separately, as it affects the existing, working GHC platforms and is straightforward to review. [edit: ghc: bootstrap 8.2.2 with 8.2.1-binary. #33350 has been merged]

• armv7l-linux won't bootstrap with ghc821binary because that build has an explicit LLVM dependency. (See https://www.haskell.org/ghc/download_ghc_8_2_1.html#linux_armv7). That is easy enough to fix and something I had done in my GHC for ARM overlays, so I will work on that next. [edit: Make GHC work on armv7l-linux #33405 fixes GHC on armv7l-linux]

• [edit: new issue] ghc822 does not bootstrap on armv7l-linux due to a segfault during stage2. mitchy on IRC is able to reproduce the exact same behavior I am seeing on NixOS on Alpine Linux, so this definitely appears to be an upstream regression. mitchy will file an issue upstream. I would like to propose that we make ghc802 the default on armv7l-linux until the issue is fixed as ghc802 is working very well for me on that platform and a good number of popular haskellPackages in master still build fine on that version of GHC. [edit: I think @peti is probably right that doing the override in nixpkgs is not necessary at this point.]

• The ghc821Binary build for aarch64-linux also has an explicit LLVM dependency, but haskellPackages for that platform won't even evaluate because apparently evaluating haskellPackages needs ghc7103Binary, even if ghc822 is bootstrapping with ghc821Binary. I haven't dug into that yet but it probably has something to do with one or more of hscolour, alex, or happy.

• It would be nice to get Hydra building at least some core haskellPackages on aarch64-linux. As far as I know, there are no armv7l-linux build hosts so support for that platform on Hydra is probably not possible.

[arianvp]

Actually there is an armv7l-linux community mirror:

http://nixos-arm.dezgeg.me/channel

See the wiki: https://nixos.wiki/wiki/NixOS_on_ARM

[dhess]

Yes, I am using @dezgeg's binary cache for armv7l-linux. However, it is not used by the nixos.org Hydra, which is what I meant here by "Hydra."

[dhess]

Update: GHC is working great now on armv7l-linux... if you are willing to use ghc802, that is. ghc822 appears to be broken upstream on that platform (see my edit to the original issue above for details).

@peti, how would you feel about making ghc802 the default on armv7l-linux? It might be some time before the issue is fixed upstream, and it would be nice if people could do some Haskell development on armv7l-linux in the meantime without needing a platform-specific override in their local configuration every time.

In any case, getting to this point has taken quite a bit more time than I expected (armv7l-linux Haskell builds are slow, even on decent hardware), so I will not be able to jump right into the aarch64-linux work right away. I will try to revisit it here and there in the coming weeks, but if someone else wants to take the baton from here, please feel free!

[peti]

How would you feel about making ghc802 the default on armv7l-linux?

I see your point, of course. At the same time, it's bound to be confusing for users if haskellPackages refers to a different major release of the compiler (and a different library ecosystem) on two different architectures. Personally, I wouldn't do it. If you feel it's the right choice, then I won't object either, though.

[dhess]

I don’t have a strong feeling one way or the other, so I defer to your judgement. If we had a bunch of armv7l-linux Hydra remote builders then I would say that the benefit of having Haskell packages built for the platform would be worth the possible confusion, given how awfully slow it is to build Haskell packages on Raspberry Pi’s and the like, but as far as I know we don’t. Maybe @dezgeg can weigh in here on how much capacity his armv7l-linux Hydra has. Also I believe that @grahamc is looking into getting some builders for the official Hydra, so if the situation changes, I would like to revisit this.

For now, though, perhaps we should just add a note to the manual or wiki so that people know it’s possible to build Haskell programs on armv7l-linux if they override the compiler.

[Ericson2314]

8.2.2 should work with cross, fwiw.

[dhess]

That sounds interesting. As of which commit? (There is still the matter of Template Haskell, of course.)

[peti]

The question which compiler haskellPackages.ghc should refer to is not connected to the question of whether or not we'll have binaries from Hydra. We can configure Hydra to build any package set we like on ARM CPUs -- it's not limited to "haskellPackages".

[Ericson2314]

@peti That's true. The idea here would be steering people towards the package set that is cached, even if that caching happens regardless. I think I lean against it though; changing things to compensate for unportability just creates more unportability.

@dhess https://hydra.nixos.org/jobset/nixpkgs/ericson2314-cross-trunk well now I made it worse, but I think I can after make it better.

[dhess]

I agree with @Ericson2314 on the Hydra matter. If we're not going to make the cached packages easy to discover by making that cached set the default haskellPackages then I think much of the benefit is lost. At the very least, we should make a semi-official remark in the manual or wiki that if you want to use Haskell on armv7l-linux, then you should set your haskellPackages to the ghc802 version, rather than expecting users just to start trying earlier versions and hope that they get lucky.

Thinking about this some more, there is also a danger that people will set an override now and then, when armv7l-linux catches up to the other platforms, they will not find out about it until months later.

Honestly, though, I suspect this affects so few people that it doesn't really matter in the big picture. Let's optimize for whatever is the best use of our volunteers' limited time (@peti, @dezgeg, etc.).

[arianvp]

I was actually trying to get haskell set up on my Odroid XU4 today (which is arm7l-linux) but not sure where to start. Would love to get hints on how to get it going and from the discussion above I don't really understand yet how (I'm not a nix veteran).

Also, the ODROID is quite a 'beefy' machine for ARM (octa-core, little-big arch) terms; perhaps I can help fill the build cache? It's got a gigabit ethernet port and I've got the ability to hook upto 2TiB of storage to it

[dhess]

@arianvp As of the time I write this, if you point your XU4 to the nixos-unstable-small channel and override/overlay haskellPackages on armv7l-linux to haskell.packages.ghc802, you should be able to build most of the widely-used Haskell packages for that platform. (The other unstable channels will soon follow.)

I've only run into a few problems so far, and most of those were solved with a dontCheck or dontHaddock override. For the packages that built successfully, I haven't had any run-time problems at all, and I have been running Haskell services on my armv7l-linux NixOS hosts for several months, using an overlay I wrote that is no longer needed now that ghc802 support is in mainline Nixpkgs.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[domenkozar]

GHC 8.8.4 now builds on aarch64 via #66277 and 8.2.2 is relatively old, I'll close this