Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Support more platforms #24

Closed
7 of 17 tasks
ErikSchierboom opened this issue Oct 9, 2020 · 13 comments
Closed
7 of 17 tasks

build: Support more platforms #24

ErikSchierboom opened this issue Oct 9, 2020 · 13 comments
Labels
kind: build Non-.nim changes that affect the release binary

Comments

@ErikSchierboom
Copy link
Member

ErikSchierboom commented Oct 9, 2020

In particular, we should be supporting:

  • Linux 32-bit
  • Windows 32-bit
  • Linux arm64
  • macOS arm64

Edit by @ee7:

@ee7 is aiming towards something like:

  • linux_x86-64
  • macos_x86-64
  • windows_x86-64

then arm64 (AKA aarch64):

then:

  • linux_powerpc64 (which is big endian) (will be added by #799)
  • linux_riscv64 (added in a962b18)

then:

Edit 2023-08-22: These 32-bit builds are possible, but I think are probably not worthwhile:

  • linux_arm
  • linux_i386
  • linux_mipsel
  • linux_powerpc
  • windows_arm
  • windows_i386

see also the Zig 0.11.0 Support Table.

@ee7 ee7 changed the title Support more platforms build: Support more platforms May 9, 2021
@ee7 ee7 added the kind: build Non-.nim changes that affect the release binary label May 10, 2021
@ee7
Copy link
Member

ee7 commented Oct 22, 2021

(Moving my post)

I was considering going via Zig for this. See https://github.com/ee7/binary-size - we can cross-compile just by writing a different target in

--passC:'-target x86_64-linux-musl' --passL:'-target x86_64-linux-musl'

and adding e.g. --cpu:arm64

See:

@ErikSchierboom
Copy link
Member Author

ErikSchierboom commented Oct 22, 2021

I was considering going via Zig for this. See https://github.com/ee7/binary-size - we can cross-compile just by writing a different target in

I'm fine with that, as long as contributors (like me) don't have to install Zig locally to build/run things. I imagine this is only something we do when building the binaries for the release?

@ee7
Copy link
Member

ee7 commented Oct 24, 2021

I imagine this is only something we do when building the binaries for the release?

Yes. But that said, on Linux, we currently require musl to be installed if a user wants to create a release build:

configlet/config.nims

Lines 10 to 20 in a103ac4

if defined(release):
switch("opt", "size")
switch("passC", "-flto")
if defined(linux) or defined(windows):
switch("passL", "-s")
switch("passL", "-static")
if defined(linux):
switch("gcc.exe", "musl-gcc")
switch("gcc.linkerexe", "musl-gcc")

Arguably, this should check if musl-gcc is actually available, but I wanted it to fail noisily if it's unavailable (rather than silently creating a non-musl-linked release).There's some tension between what we want for a developer, and what we want for a build job.

I plan to start our Zig-ing by making the release build go via Zig if it's available. This will also upgrade our end-of-life musl version (resolving #194).

As an aside, it also reduces the release build time significantly, as it currently does better caching (--passC:-flto makes the release build take a "long time"). Roughly 10s -> 2s on a typical machine for a rebuild.

@ErikSchierboom
Copy link
Member Author

Sounds good!

@ee7
Copy link
Member

ee7 commented Apr 11, 2022

All we can say from the current configlet download data is that it's downloaded about 3x more often on macOS than Windows. We can't easily know the Linux numbers, since the vast majority are CI downloads.

But it's interesting to look at the stats for the most recent release of exercism/cli, which gives us download data from the last 2.5 years.

gh release view v3.0.13 --repo exercism/cli --json assets \
  | jq -r '.assets | .[] | [.name,.downloadCount] | @tsv' \
  | column -t -N 'Asset name,Download count' -R 2 -o ' | '
Asset name Download count
exercism-3.0.13-darwin-i386.tar.gz 1613
exercism-3.0.13-darwin-x86_64.tar.gz 3187
exercism-3.0.13-freebsd-i386.tar.gz 56
exercism-3.0.13-freebsd-x86_64.tar.gz 275
exercism-3.0.13-linux-armv5.tar.gz 211
exercism-3.0.13-linux-armv6.tar.gz 726
exercism-3.0.13-linux-i386.tar.gz 342
exercism-3.0.13-linux-ppc64.tar.gz 153
exercism-3.0.13-linux-x86_64.tar.gz 24161
exercism-3.0.13-openbsd-i386.tar.gz 52
exercism-3.0.13-openbsd-x86_64.tar.gz 212
exercism-3.0.13-windows-i386.zip 683
exercism-3.0.13-windows-x86_64.zip 7053
exercism-freebsd-32bit.tgz 44
exercism-freebsd-64bit.tgz 155
exercism-linux-32bit.tgz 225
exercism-linux-64bit.tgz 21820
exercism-linux-ppc64.tgz 70
exercism-mac-32bit.tgz 191
exercism-mac-64bit.tgz 3938
exercism-openbsd-32bit.tgz 33
exercism-openbsd-64bit.tgz 218
exercism-windows-32bit.zip 1907
exercism-windows-64bit.zip 115582
exercism_checksums.txt 631
exercism_checksums.txt.sig 126

After combining duplicate release data (the release naming convention was changed, but releases with the old convention were also uploaded for backward compatibility), then sorting:

Operating System Architecture Download count
Windows x86_64 122635
Linux x86_64 45981
macOS x86_64 7125
Windows i386 2590
macOS i386 1804
Linux armv6 726
Linux i386 567
FreeBSD x86_64 430
OpenBSD x86_64 430
Linux ppc64 223
Linux armv5 211
FreeBSD i386 100
OpenBSD i386 85

I imagine that:

  • The OS/Arch data for "people who use configlet" is vastly different from "people who use the Exercism CLI".
  • The Exercism CLI usage changed drastically after the release of Exercism v3, and some types of user are more likely to keep using the CLI.

But still... interesting.

I'm hoping that we can release configlet on more platforms than the CLI (adding arm64 on Linux and macOS at least), but if there's some unforeseen reason that we have to prioritise certain platforms, it might be worth keeping the above data in mind.

Note that I have no plans to set up CI pipelines for platforms that GitHub Actions do not provide - the others will be cross-compiled and largely untested (although we could use https://github.com/docker/setup-qemu-action). Azure got arm64 recently, so GitHub Actions might get that eventually too. Otherwise, BSDs are available on sourcehut, and ARM/ARM64 are available on Drone CI and CircleCI. (And in the days before GitHub Actions, I used to use arm64, ppc64, and s390x on Travis CI).

Let me know if you have any thoughts.

@ErikSchierboom
Copy link
Member Author

adding arm64 on Linux and macOS at least

Agreed. Those seem to be the ones that would benefit most from having a specific build.

Note that I have no plans to set up CI pipelines for platforms that GitHub Actions do not provide

I agree with that. We can put in a caveat in the README or something that it is untested, but with the current complexity of the code, I don't think it is likely that there'll be many issues. I'd prefer not to have different CI platforms.

@kotp
Copy link
Member

kotp commented Apr 17, 2022

Anyway to measure builds? Those that compile the CLI instead of using a pre-compiled executable?

@ee7
Copy link
Member

ee7 commented Apr 17, 2022

Anyway to measure builds? Those that compile the CLI instead of using a pre-compiled executable?

If you mean "can we know how many times has go build been run to compile exercism/cli?", or "how many people are using the CLI without using a CLI release binary?" then I think the answer is "no". I imagine that the build process for the CLI release binaries doesn't do something to make those binaries distinguishable later from self-built ones, but I could be wrong about that.

But there is both a binary and source package in e.g. the AUR:

There are some git clone stats at https://github.com/exercism/cli/graphs/traffic for those with certain privileges in that repo, but the stats are only for the previous 2 weeks, and they might be too noisy to be useful.

And AFAIK, GitHub does not make download stats available for the source archives (e.g. https://github.com/exercism/cli/archive/refs/tags/v3.0.13.tar.gz).

@kotp
Copy link
Member

kotp commented Apr 22, 2022

There could be a "call home" trigger in the build that could report this. Not sure if it is worthwhile doing though.

@ee7
Copy link
Member

ee7 commented Oct 26, 2022

I think the plan should be something like:

First, get x86-64 and arm64 on Linux/macOS/Windows:

configlet_4.1.0_linux_arm64.tar.gz
configlet_4.1.0_linux_x86-64.tar.gz
configlet_4.1.0_macos_arm64.tar.gz
configlet_4.1.0_macos_x86-64.tar.gz
configlet_4.1.0_windows_arm64.zip
configlet_4.1.0_windows_x86-64.zip

Then add BSDs, and more 64-bit Linux builds:

configlet_4.2.0_freebsd_x86-64.tar.gz
configlet_4.2.0_linux_powerpc64.tar.gz
configlet_4.2.0_linux_riscv64.tar.gz
configlet_4.2.0_netbsd_x86-64.tar.gz
configlet_4.2.0_openbsd_x86-64.tar.gz

Then consider adding 32-bit assets, like:

configlet_4.3.0_linux_arm.tar.gz
configlet_4.3.0_linux_i386.tar.gz
configlet_4.3.0_windows_i386.zip

(Where the version numbers have no particular meaning).

ee7 added a commit that referenced this issue Nov 11, 2022
Before this commit, the release assets were named like:

    configlet-linux-64bit.tgz
    configlet-mac-64bit.tgz
    configlet-windows-64bit.zip
    configlet_4.0.0-beta.7_checksums_sha256.txt

With this commit, the next release will have assets named:

    configlet_4.0.0-beta.8_checksums_sha256.txt
    configlet_4.0.0-beta.8_linux_x86-64.tar.gz
    configlet_4.0.0-beta.8_macos_x86-64.tar.gz
    configlet_4.0.0-beta.8_windows_x86-64.zip

Where we:

- Make the archive naming format match that of the checksums file
- Add a version string
- Delimit with an underscore, not a hyphen
- Rename `64bit` to `x86-64`
- Rename `32bit` to `i386`
- Rename `tgz` to `tar.gz`
- Rename `mac` to `macos`

So far, we have only released configlet for x86-64, and the release
assets have always had an ambiguous `64bit` in the names. This naming
format was old (added by d4c6e26, 2020-10-09), and was ultimately
inherited from:

- the `exercism/configlet-v2` releases [1]
- the `exercism/cli` releases [2][3]

However, we're getting closer to adding releases for other 64-bit
architectures, so it's especially important to change the names now.

The rationale for the particular use of underscores and hyphens is:

- We want to have the version string in the asset filenames

- Configlet uses version strings that are compatible with the Semantic
  Versioning spec

- It is more common to use hyphens in executable names than
  underscores

- The Semantic Versioning spec forbids using an underscore, and
  specifies that prerelease versions can use hyphens [4]:

      Section 9:

      A pre-release version MAY be denoted by appending a hyphen and a
      series of dot separated identifiers immediately following the patch
      version. Identifiers MUST comprise only ASCII alphanumerics and
      hyphens [0-9A-Za-z-].
      [...]
      Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92

That is, there's an argument for the format of:

    some-app_1.0.0-beta.1_linux_x86-64.tar.gz

rather than:

    some_app-1.0.0-beta.1-linux-x86_64.tar.gz

because `_` fully separates the components of the first, and `-` does
not separate the components of the second (due to the `-beta` part of
the version string). This does ignore the convention of `_` as an
inter-word space, however.

[1] https://github.com/exercism/v2-configlet/releases
[2] exercism/cli#700 (comment)
[3] https://github.com/exercism/cli/releases/tag/v3.0.12
[4] https://semver.org/

Closes: #363
Refs: #24
petertseng pushed a commit to petertseng/exercism-rust that referenced this issue Nov 17, 2022
Before this commit, the release assets were named like:

    configlet-linux-64bit.tgz
    configlet-mac-64bit.tgz
    configlet-windows-64bit.zip
    configlet_4.0.0-beta.7_checksums_sha256.txt

With this commit, the next release will have assets named:

    configlet_4.0.0-beta.8_checksums_sha256.txt
    configlet_4.0.0-beta.8_linux_x86-64.tar.gz
    configlet_4.0.0-beta.8_macos_x86-64.tar.gz
    configlet_4.0.0-beta.8_windows_x86-64.zip

Where we:

- Make the archive naming format match that of the checksums file
- Add a version string
- Delimit with an underscore, not a hyphen
- Rename `64bit` to `x86-64`
- Rename `32bit` to `i386`
- Rename `tgz` to `tar.gz`
- Rename `mac` to `macos`

So far, we have only released configlet for x86-64, and the release
assets have always had an ambiguous `64bit` in the names. This naming
format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately
inherited from:

- the `exercism/configlet-v2` releases [1]
- the `exercism/cli` releases [2][3]

However, we're getting closer to adding releases for other 64-bit
architectures, so it's especially important to change the names now.

The rationale for the particular use of underscores and hyphens is:

- We want to have the version string in the asset filenames

- Configlet uses version strings that are compatible with the Semantic
  Versioning spec

- It is more common to use hyphens in executable names than
  underscores

- The Semantic Versioning spec forbids using an underscore, and
  specifies that prerelease versions can use hyphens [4]:

      Section 9:

      A pre-release version MAY be denoted by appending a hyphen and a
      series of dot separated identifiers immediately following the patch
      version. Identifiers MUST comprise only ASCII alphanumerics and
      hyphens [0-9A-Za-z-].
      [...]
      Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92

That is, there's an argument for the format of:

    some-app_1.0.0-beta.1_linux_x86-64.tar.gz

rather than:

    some_app-1.0.0-beta.1-linux-x86_64.tar.gz

because `_` fully separates the components of the first, and `-` does
not separate the components of the second (due to the `-beta` part of
the version string). This does ignore the convention of `_` as an
inter-word space, however.

[1] https://github.com/exercism/v2-configlet/releases
[2] exercism/cli#700 (comment)
[3] https://github.com/exercism/cli/releases/tag/v3.0.12
[4] https://semver.org/

Closes: exercism/configlet#363
Refs: exercism/configlet#24
exercism/configlet#705
petertseng pushed a commit to petertseng/exercism-haskell that referenced this issue Nov 17, 2022
Before this commit, the release assets were named like:

    configlet-linux-64bit.tgz
    configlet-mac-64bit.tgz
    configlet-windows-64bit.zip
    configlet_4.0.0-beta.7_checksums_sha256.txt

With this commit, the next release will have assets named:

    configlet_4.0.0-beta.8_checksums_sha256.txt
    configlet_4.0.0-beta.8_linux_x86-64.tar.gz
    configlet_4.0.0-beta.8_macos_x86-64.tar.gz
    configlet_4.0.0-beta.8_windows_x86-64.zip

Where we:

- Make the archive naming format match that of the checksums file
- Add a version string
- Delimit with an underscore, not a hyphen
- Rename `64bit` to `x86-64`
- Rename `32bit` to `i386`
- Rename `tgz` to `tar.gz`
- Rename `mac` to `macos`

So far, we have only released configlet for x86-64, and the release
assets have always had an ambiguous `64bit` in the names. This naming
format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately
inherited from:

- the `exercism/configlet-v2` releases [1]
- the `exercism/cli` releases [2][3]

However, we're getting closer to adding releases for other 64-bit
architectures, so it's especially important to change the names now.

The rationale for the particular use of underscores and hyphens is:

- We want to have the version string in the asset filenames

- Configlet uses version strings that are compatible with the Semantic
  Versioning spec

- It is more common to use hyphens in executable names than
  underscores

- The Semantic Versioning spec forbids using an underscore, and
  specifies that prerelease versions can use hyphens [4]:

      Section 9:

      A pre-release version MAY be denoted by appending a hyphen and a
      series of dot separated identifiers immediately following the patch
      version. Identifiers MUST comprise only ASCII alphanumerics and
      hyphens [0-9A-Za-z-].
      [...]
      Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92

That is, there's an argument for the format of:

    some-app_1.0.0-beta.1_linux_x86-64.tar.gz

rather than:

    some_app-1.0.0-beta.1-linux-x86_64.tar.gz

because `_` fully separates the components of the first, and `-` does
not separate the components of the second (due to the `-beta` part of
the version string). This does ignore the convention of `_` as an
inter-word space, however.

[1] https://github.com/exercism/v2-configlet/releases
[2] exercism/cli#700 (comment)
[3] https://github.com/exercism/cli/releases/tag/v3.0.12
[4] https://semver.org/

Closes: exercism/configlet#363
Refs: exercism/configlet#24
exercism/configlet#705
petertseng pushed a commit to petertseng/exercism-haskell that referenced this issue Nov 17, 2022
Before this commit, the release assets were named like:

    configlet-linux-64bit.tgz
    configlet-mac-64bit.tgz
    configlet-windows-64bit.zip
    configlet_4.0.0-beta.7_checksums_sha256.txt

With this commit, the next release will have assets named:

    configlet_4.0.0-beta.8_checksums_sha256.txt
    configlet_4.0.0-beta.8_linux_x86-64.tar.gz
    configlet_4.0.0-beta.8_macos_x86-64.tar.gz
    configlet_4.0.0-beta.8_windows_x86-64.zip

Where we:

- Make the archive naming format match that of the checksums file
- Add a version string
- Delimit with an underscore, not a hyphen
- Rename `64bit` to `x86-64`
- Rename `32bit` to `i386`
- Rename `tgz` to `tar.gz`
- Rename `mac` to `macos`

So far, we have only released configlet for x86-64, and the release
assets have always had an ambiguous `64bit` in the names. This naming
format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately
inherited from:

- the `exercism/configlet-v2` releases [1]
- the `exercism/cli` releases [2][3]

However, we're getting closer to adding releases for other 64-bit
architectures, so it's especially important to change the names now.

The rationale for the particular use of underscores and hyphens is:

- We want to have the version string in the asset filenames

- Configlet uses version strings that are compatible with the Semantic
  Versioning spec

- It is more common to use hyphens in executable names than
  underscores

- The Semantic Versioning spec forbids using an underscore, and
  specifies that prerelease versions can use hyphens [4]:

      Section 9:

      A pre-release version MAY be denoted by appending a hyphen and a
      series of dot separated identifiers immediately following the patch
      version. Identifiers MUST comprise only ASCII alphanumerics and
      hyphens [0-9A-Za-z-].
      [...]
      Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92

That is, there's an argument for the format of:

    some-app_1.0.0-beta.1_linux_x86-64.tar.gz

rather than:

    some_app-1.0.0-beta.1-linux-x86_64.tar.gz

because `_` fully separates the components of the first, and `-` does
not separate the components of the second (due to the `-beta` part of
the version string). This does ignore the convention of `_` as an
inter-word space, however.

[1] https://github.com/exercism/v2-configlet/releases
[2] exercism/cli#700 (comment)
[3] https://github.com/exercism/cli/releases/tag/v3.0.12
[4] https://semver.org/

exercism/configlet#363
exercism/configlet#24
exercism/configlet#705
petertseng pushed a commit to petertseng/exercism-rust that referenced this issue Nov 17, 2022
Before this commit, the release assets were named like:

    configlet-linux-64bit.tgz
    configlet-mac-64bit.tgz
    configlet-windows-64bit.zip
    configlet_4.0.0-beta.7_checksums_sha256.txt

With this commit, the next release will have assets named:

    configlet_4.0.0-beta.8_checksums_sha256.txt
    configlet_4.0.0-beta.8_linux_x86-64.tar.gz
    configlet_4.0.0-beta.8_macos_x86-64.tar.gz
    configlet_4.0.0-beta.8_windows_x86-64.zip

Where we:

- Make the archive naming format match that of the checksums file
- Add a version string
- Delimit with an underscore, not a hyphen
- Rename `64bit` to `x86-64`
- Rename `32bit` to `i386`
- Rename `tgz` to `tar.gz`
- Rename `mac` to `macos`

So far, we have only released configlet for x86-64, and the release
assets have always had an ambiguous `64bit` in the names. This naming
format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately
inherited from:

- the `exercism/configlet-v2` releases [1]
- the `exercism/cli` releases [2][3]

However, we're getting closer to adding releases for other 64-bit
architectures, so it's especially important to change the names now.

The rationale for the particular use of underscores and hyphens is:

- We want to have the version string in the asset filenames

- Configlet uses version strings that are compatible with the Semantic
  Versioning spec

- It is more common to use hyphens in executable names than
  underscores

- The Semantic Versioning spec forbids using an underscore, and
  specifies that prerelease versions can use hyphens [4]:

      Section 9:

      A pre-release version MAY be denoted by appending a hyphen and a
      series of dot separated identifiers immediately following the patch
      version. Identifiers MUST comprise only ASCII alphanumerics and
      hyphens [0-9A-Za-z-].
      [...]
      Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92

That is, there's an argument for the format of:

    some-app_1.0.0-beta.1_linux_x86-64.tar.gz

rather than:

    some_app-1.0.0-beta.1-linux-x86_64.tar.gz

because `_` fully separates the components of the first, and `-` does
not separate the components of the second (due to the `-beta` part of
the version string). This does ignore the convention of `_` as an
inter-word space, however.

[1] https://github.com/exercism/v2-configlet/releases
[2] exercism/cli#700 (comment)
[3] https://github.com/exercism/cli/releases/tag/v3.0.12
[4] https://semver.org/

exercism/configlet#363
exercism/configlet#24
exercism/configlet#705
ee7 added a commit that referenced this issue Aug 16, 2023
Add a script to install Zig, and add a job to the build workflow that
uses `zig cc` to cross compile configlet for arm64 (AKA aarch64) Linux.

This means that the next configlet release will have two new release
assets:

    configlet_4.0.0-beta.14_linux_arm64.tar.gz
    configlet_4.0.0-beta.14_linux_arm64.tar.gz.minisig

and extracting the archive will yield the executable:

    $ file ./configlet
    ./configlet: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped

Also support running `nimble build -d:zig` locally.

For now, the new executable has a large comment section:

    $ readelf -p .comment ./configlet

    String dump of section '.comment':
      [     0]  clang version 16.0.6 (https://github.com/ziglang/zig-bootstrap 1dda86241204c4649f668d46b6a37feed707c7b4)
      [...] (repeat for 39 KiB)
      [  8d82]  clang version 16.0.6 (https://github.com/ziglang/zig-bootstrap 1dda86241204c4649f668d46b6a37feed707c7b4)
      [  8deb]  Linker: LLD 16.0.6

Try to strip it later. Our `strip` doesn't support elf64-aarch64 as a
target, and `zig objcopy` doesn't have a `-R, --remove-section` option
currently. But prepare to use `llvm-strip`.

Cross-compiling for arm64 macos currently produces an error:

    CC: ../nimdir/lib/std/sysrand.nim
    /home/runner/.cache/nim/configlet_r/@m..@snimdir@slib@sstd@ssysrand.nim.c:9:10:
    fatal error: 'Security/SecRandom.h' file not found

which is due to a recent commit [1].

See some posts on cross-compiling with Zig [2][3][4] and the support table [5].

Refs: #24
Refs: #122
Refs: #764

[1] 53a75a2 ("nimble, uuid: generate UUIDs via std/sysrand, not pragmagic/uuids", 2023-08-07)
[2] https://ziglang.org/learn/overview/#cross-compiling-is-a-first-class-use-case
[3] https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html
[4] https://www.uber.com/blog/bootstrapping-ubers-infrastructure-on-arm64-with-zig/
[5] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
ee7 added a commit that referenced this issue Aug 17, 2023
Continue the recent `zig cc` work [1], such that the next configlet
release will have two new release assets:

    configlet_4.0.0-beta.14_macos_arm64.tar.gz
    configlet_4.0.0-beta.14_macos_arm64.tar.gz.minisig

where the archive contains the executable:

    $ file ./configlet
    configlet: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|HAS_TLV_DESCRIPTORS>

`configlet uuid` now uses [2] `std/sysrand`, which on macOS uses [3] the
Security Framework. So to produce the macOS arm64 configlet, it's
easiest to cross-compile from macOS x86_64, where the macOSX SDK is
already available. Make the cross-compile and install-zig scripts
support macOS, and add a build job for it.

It looks like the macos-12 GitHub runner has SDK versions 12.3 and 13.1
installed. Use the latest one.

The new executable is about 1.09 MiB, which is notably larger than the
655 KiB macOS x86_64 executable. Possible ways to reduce the size in
the future:

- Enable LTO when `zig ld` supports it [4].

- Compile it natively, when GitHub begins to provide a hosted arm64
  macOS runner [5].

- Compile it natively, using a non-GitHub arm64 macOS runner.

Refs: #24
Refs: #122
Refs: #764

[1] 0e8d665, ".github, config: use Zig to cross-compile arm64 Linux asset", 2023-08-13
[2] 53a75a2, "nimble, uuid: generate UUIDs via std/sysrand, not pragmagic/uuids", 2023-08-07
[3] https://github.com/nim-lang/Nim/blob/v2.0.0/lib/std/sysrand.nim#L235-L256
[4] https://www.github.com/ziglang/zig/issues/8680
[5] https://www.github.com/github/roadmap/issues/528
ee7 added a commit that referenced this issue Aug 17, 2023
Continue the recent `zig cc` work [1], such that the next configlet
release will have two new release assets:

    configlet_4.0.0-beta.14_linux_riscv64.tar.gz
    configlet_4.0.0-beta.14_linux_riscv64.tar.gz.minisig

where the archive contains the executable:

    $ file ./configlet
    ./configlet: ELF 64-bit LSB executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), statically linked, stripped

The riscv64-linux-musl target will have Tier 1 Zig support [2].

Refs: #24

[1] 0e8d665, ".github, config: use Zig to cross-compile arm64 Linux asset", 2023-08-13
[2] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
ee7 added a commit that referenced this issue Aug 18, 2023
Continue the recent `zig cc` work [1][2][3], such that the next
configlet release will have two new release assets:

    configlet_4.0.0-beta.14_windows_arm64.zip
    configlet_4.0.0-beta.14_windows_arm64.zip.minisig

where the archive contains the executable:

    $ file ./configlet.exe
    ./configlet: PE32+ executable (console) Aarch64, for MS Windows, 6 sections

The aarch64-windows-gnu target will have Tier 1 Zig support [4].

Also make the cross-compilation jobs and scripts work on Windows, even
though we currently cross-compile the new executable from Linux.

Refs: #24
Refs: #122
Closes: #764

[1] 0e8d665, 2023-08-16, ".github, config: use Zig to cross-compile arm64 Linux asset"
[2] f280445, 2023-08-17, ".github: cross-compile arm64 macOS asset"
[3] a962b18, 2023-08-17, ".github: cross-compile riscv64 Linux asset"
[4] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
@ee7
Copy link
Member

ee7 commented Aug 22, 2023

@ErikSchierboom you wrote (albeit nearly 3 years ago):

In particular, we should be supporting:

  • Linux 32-bit
  • Windows 32-bit

Did this mean:

  1. x86
  2. arm
  3. x86 and arm
  4. or x86, arm, plus others?

And how much do we care about this? Was it just motivated by the assets we had for v2-configlet?

It looks like zig cc will eventually have Tier 1 support for:

  • arm-windows-gnu
  • arm-linux-musleabi
  • mipsel-linux-musl
  • powerpc-linux-musl
  • x86-linux-musl
  • x86-windows-gnu

But I'm not keen on providing builds for them because:

  • I suspect that the number of people who currently (or in the future) want to run configlet on 32-bit platforms is essentially 0.
  • We'd need some dedicated changes, because e.g. some prob-specs JSON files (e.g. for grains) contain numbers that exceed the maximum value of a 32-bit int.
  • It's simplest for configlet to have the absolute minimum amount of architecture-specific code.
  • It's questionable to provide those executables pre-built if we don't have integration tests for them.
  • Adding 32-bit integration testing will take a little bit of work (but probably not much if we gain qemu integration testing of the existing cross-compiled executables).
  • It'd be better to spend time on any other configlet issue.

Thoughts?

Adding further platforms isn't a priority for me. But if I do it for my own interest, next would be x86_64 BSDs (I've built and tested configlet on them in the past) and more 64-bit Linux builds (e.g. powerpc64).

@ErikSchierboom
Copy link
Member Author

And how much do we care about this? Was it just motivated by the assets we had for v2-configlet?

It was, so I'm happy to ditch that. We've also hadn't really had people requesting 32-bit support AFAICT so 🤷

@ee7
Copy link
Member

ee7 commented Aug 22, 2023

Okay, then I'll suggest that this issue is closed by our recent adding of arm64 assets in these commits:

I'll open separate issues to track some possible future additions like the BSDs, but won't open issues for 32-bit targets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: build Non-.nim changes that affect the release binary
Projects
Status: Done
Development

No branches or pull requests

3 participants