-
-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: Support more platforms #24
Comments
(Moving my post) I was considering going via Zig for this. See https://github.com/ee7/binary-size - we can cross-compile just by writing a different target in
and adding e.g. See:
|
I'm fine with that, as long as contributors (like me) don't have to install Zig locally to build/run things. I imagine this is only something we do when building the binaries for the release? |
Yes. But that said, on Linux, we currently require musl to be installed if a user wants to create a release build: Lines 10 to 20 in a103ac4
Arguably, this should check if I plan to start our Zig-ing by making the release build go via Zig if it's available. This will also upgrade our end-of-life musl version (resolving #194). As an aside, it also reduces the release build time significantly, as it currently does better caching ( |
Sounds good! |
All we can say from the current configlet download data is that it's downloaded about 3x more often on macOS than Windows. We can't easily know the Linux numbers, since the vast majority are CI downloads. But it's interesting to look at the stats for the most recent release of gh release view v3.0.13 --repo exercism/cli --json assets \
| jq -r '.assets | .[] | [.name,.downloadCount] | @tsv' \
| column -t -N 'Asset name,Download count' -R 2 -o ' | '
After combining duplicate release data (the release naming convention was changed, but releases with the old convention were also uploaded for backward compatibility), then sorting:
I imagine that:
But still... interesting. I'm hoping that we can release configlet on more platforms than the CLI (adding arm64 on Linux and macOS at least), but if there's some unforeseen reason that we have to prioritise certain platforms, it might be worth keeping the above data in mind. Note that I have no plans to set up CI pipelines for platforms that GitHub Actions do not provide - the others will be cross-compiled and largely untested (although we could use https://github.com/docker/setup-qemu-action). Azure got arm64 recently, so GitHub Actions might get that eventually too. Otherwise, BSDs are available on sourcehut, and ARM/ARM64 are available on Drone CI and CircleCI. (And in the days before GitHub Actions, I used to use arm64, ppc64, and s390x on Travis CI). Let me know if you have any thoughts. |
Agreed. Those seem to be the ones that would benefit most from having a specific build.
I agree with that. We can put in a caveat in the README or something that it is untested, but with the current complexity of the code, I don't think it is likely that there'll be many issues. I'd prefer not to have different CI platforms. |
Anyway to measure builds? Those that compile the CLI instead of using a pre-compiled executable? |
If you mean "can we know how many times has But there is both a binary and source package in e.g. the AUR:
There are some And AFAIK, GitHub does not make download stats available for the source archives (e.g. https://github.com/exercism/cli/archive/refs/tags/v3.0.13.tar.gz). |
There could be a "call home" trigger in the build that could report this. Not sure if it is worthwhile doing though. |
I think the plan should be something like: First, get
Then add BSDs, and more 64-bit Linux builds:
Then consider adding 32-bit assets, like:
(Where the version numbers have no particular meaning). |
Before this commit, the release assets were named like: configlet-linux-64bit.tgz configlet-mac-64bit.tgz configlet-windows-64bit.zip configlet_4.0.0-beta.7_checksums_sha256.txt With this commit, the next release will have assets named: configlet_4.0.0-beta.8_checksums_sha256.txt configlet_4.0.0-beta.8_linux_x86-64.tar.gz configlet_4.0.0-beta.8_macos_x86-64.tar.gz configlet_4.0.0-beta.8_windows_x86-64.zip Where we: - Make the archive naming format match that of the checksums file - Add a version string - Delimit with an underscore, not a hyphen - Rename `64bit` to `x86-64` - Rename `32bit` to `i386` - Rename `tgz` to `tar.gz` - Rename `mac` to `macos` So far, we have only released configlet for x86-64, and the release assets have always had an ambiguous `64bit` in the names. This naming format was old (added by d4c6e26, 2020-10-09), and was ultimately inherited from: - the `exercism/configlet-v2` releases [1] - the `exercism/cli` releases [2][3] However, we're getting closer to adding releases for other 64-bit architectures, so it's especially important to change the names now. The rationale for the particular use of underscores and hyphens is: - We want to have the version string in the asset filenames - Configlet uses version strings that are compatible with the Semantic Versioning spec - It is more common to use hyphens in executable names than underscores - The Semantic Versioning spec forbids using an underscore, and specifies that prerelease versions can use hyphens [4]: Section 9: A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. [...] Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92 That is, there's an argument for the format of: some-app_1.0.0-beta.1_linux_x86-64.tar.gz rather than: some_app-1.0.0-beta.1-linux-x86_64.tar.gz because `_` fully separates the components of the first, and `-` does not separate the components of the second (due to the `-beta` part of the version string). This does ignore the convention of `_` as an inter-word space, however. [1] https://github.com/exercism/v2-configlet/releases [2] exercism/cli#700 (comment) [3] https://github.com/exercism/cli/releases/tag/v3.0.12 [4] https://semver.org/ Closes: #363 Refs: #24
Before this commit, the release assets were named like: configlet-linux-64bit.tgz configlet-mac-64bit.tgz configlet-windows-64bit.zip configlet_4.0.0-beta.7_checksums_sha256.txt With this commit, the next release will have assets named: configlet_4.0.0-beta.8_checksums_sha256.txt configlet_4.0.0-beta.8_linux_x86-64.tar.gz configlet_4.0.0-beta.8_macos_x86-64.tar.gz configlet_4.0.0-beta.8_windows_x86-64.zip Where we: - Make the archive naming format match that of the checksums file - Add a version string - Delimit with an underscore, not a hyphen - Rename `64bit` to `x86-64` - Rename `32bit` to `i386` - Rename `tgz` to `tar.gz` - Rename `mac` to `macos` So far, we have only released configlet for x86-64, and the release assets have always had an ambiguous `64bit` in the names. This naming format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately inherited from: - the `exercism/configlet-v2` releases [1] - the `exercism/cli` releases [2][3] However, we're getting closer to adding releases for other 64-bit architectures, so it's especially important to change the names now. The rationale for the particular use of underscores and hyphens is: - We want to have the version string in the asset filenames - Configlet uses version strings that are compatible with the Semantic Versioning spec - It is more common to use hyphens in executable names than underscores - The Semantic Versioning spec forbids using an underscore, and specifies that prerelease versions can use hyphens [4]: Section 9: A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. [...] Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92 That is, there's an argument for the format of: some-app_1.0.0-beta.1_linux_x86-64.tar.gz rather than: some_app-1.0.0-beta.1-linux-x86_64.tar.gz because `_` fully separates the components of the first, and `-` does not separate the components of the second (due to the `-beta` part of the version string). This does ignore the convention of `_` as an inter-word space, however. [1] https://github.com/exercism/v2-configlet/releases [2] exercism/cli#700 (comment) [3] https://github.com/exercism/cli/releases/tag/v3.0.12 [4] https://semver.org/ Closes: exercism/configlet#363 Refs: exercism/configlet#24 exercism/configlet#705
Before this commit, the release assets were named like: configlet-linux-64bit.tgz configlet-mac-64bit.tgz configlet-windows-64bit.zip configlet_4.0.0-beta.7_checksums_sha256.txt With this commit, the next release will have assets named: configlet_4.0.0-beta.8_checksums_sha256.txt configlet_4.0.0-beta.8_linux_x86-64.tar.gz configlet_4.0.0-beta.8_macos_x86-64.tar.gz configlet_4.0.0-beta.8_windows_x86-64.zip Where we: - Make the archive naming format match that of the checksums file - Add a version string - Delimit with an underscore, not a hyphen - Rename `64bit` to `x86-64` - Rename `32bit` to `i386` - Rename `tgz` to `tar.gz` - Rename `mac` to `macos` So far, we have only released configlet for x86-64, and the release assets have always had an ambiguous `64bit` in the names. This naming format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately inherited from: - the `exercism/configlet-v2` releases [1] - the `exercism/cli` releases [2][3] However, we're getting closer to adding releases for other 64-bit architectures, so it's especially important to change the names now. The rationale for the particular use of underscores and hyphens is: - We want to have the version string in the asset filenames - Configlet uses version strings that are compatible with the Semantic Versioning spec - It is more common to use hyphens in executable names than underscores - The Semantic Versioning spec forbids using an underscore, and specifies that prerelease versions can use hyphens [4]: Section 9: A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. [...] Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92 That is, there's an argument for the format of: some-app_1.0.0-beta.1_linux_x86-64.tar.gz rather than: some_app-1.0.0-beta.1-linux-x86_64.tar.gz because `_` fully separates the components of the first, and `-` does not separate the components of the second (due to the `-beta` part of the version string). This does ignore the convention of `_` as an inter-word space, however. [1] https://github.com/exercism/v2-configlet/releases [2] exercism/cli#700 (comment) [3] https://github.com/exercism/cli/releases/tag/v3.0.12 [4] https://semver.org/ Closes: exercism/configlet#363 Refs: exercism/configlet#24 exercism/configlet#705
Before this commit, the release assets were named like: configlet-linux-64bit.tgz configlet-mac-64bit.tgz configlet-windows-64bit.zip configlet_4.0.0-beta.7_checksums_sha256.txt With this commit, the next release will have assets named: configlet_4.0.0-beta.8_checksums_sha256.txt configlet_4.0.0-beta.8_linux_x86-64.tar.gz configlet_4.0.0-beta.8_macos_x86-64.tar.gz configlet_4.0.0-beta.8_windows_x86-64.zip Where we: - Make the archive naming format match that of the checksums file - Add a version string - Delimit with an underscore, not a hyphen - Rename `64bit` to `x86-64` - Rename `32bit` to `i386` - Rename `tgz` to `tar.gz` - Rename `mac` to `macos` So far, we have only released configlet for x86-64, and the release assets have always had an ambiguous `64bit` in the names. This naming format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately inherited from: - the `exercism/configlet-v2` releases [1] - the `exercism/cli` releases [2][3] However, we're getting closer to adding releases for other 64-bit architectures, so it's especially important to change the names now. The rationale for the particular use of underscores and hyphens is: - We want to have the version string in the asset filenames - Configlet uses version strings that are compatible with the Semantic Versioning spec - It is more common to use hyphens in executable names than underscores - The Semantic Versioning spec forbids using an underscore, and specifies that prerelease versions can use hyphens [4]: Section 9: A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. [...] Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92 That is, there's an argument for the format of: some-app_1.0.0-beta.1_linux_x86-64.tar.gz rather than: some_app-1.0.0-beta.1-linux-x86_64.tar.gz because `_` fully separates the components of the first, and `-` does not separate the components of the second (due to the `-beta` part of the version string). This does ignore the convention of `_` as an inter-word space, however. [1] https://github.com/exercism/v2-configlet/releases [2] exercism/cli#700 (comment) [3] https://github.com/exercism/cli/releases/tag/v3.0.12 [4] https://semver.org/ exercism/configlet#363 exercism/configlet#24 exercism/configlet#705
Before this commit, the release assets were named like: configlet-linux-64bit.tgz configlet-mac-64bit.tgz configlet-windows-64bit.zip configlet_4.0.0-beta.7_checksums_sha256.txt With this commit, the next release will have assets named: configlet_4.0.0-beta.8_checksums_sha256.txt configlet_4.0.0-beta.8_linux_x86-64.tar.gz configlet_4.0.0-beta.8_macos_x86-64.tar.gz configlet_4.0.0-beta.8_windows_x86-64.zip Where we: - Make the archive naming format match that of the checksums file - Add a version string - Delimit with an underscore, not a hyphen - Rename `64bit` to `x86-64` - Rename `32bit` to `i386` - Rename `tgz` to `tar.gz` - Rename `mac` to `macos` So far, we have only released configlet for x86-64, and the release assets have always had an ambiguous `64bit` in the names. This naming format was old (added by d4c6e26836a5, 2020-10-09), and was ultimately inherited from: - the `exercism/configlet-v2` releases [1] - the `exercism/cli` releases [2][3] However, we're getting closer to adding releases for other 64-bit architectures, so it's especially important to change the names now. The rationale for the particular use of underscores and hyphens is: - We want to have the version string in the asset filenames - Configlet uses version strings that are compatible with the Semantic Versioning spec - It is more common to use hyphens in executable names than underscores - The Semantic Versioning spec forbids using an underscore, and specifies that prerelease versions can use hyphens [4]: Section 9: A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. [...] Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92 That is, there's an argument for the format of: some-app_1.0.0-beta.1_linux_x86-64.tar.gz rather than: some_app-1.0.0-beta.1-linux-x86_64.tar.gz because `_` fully separates the components of the first, and `-` does not separate the components of the second (due to the `-beta` part of the version string). This does ignore the convention of `_` as an inter-word space, however. [1] https://github.com/exercism/v2-configlet/releases [2] exercism/cli#700 (comment) [3] https://github.com/exercism/cli/releases/tag/v3.0.12 [4] https://semver.org/ exercism/configlet#363 exercism/configlet#24 exercism/configlet#705
Add a script to install Zig, and add a job to the build workflow that uses `zig cc` to cross compile configlet for arm64 (AKA aarch64) Linux. This means that the next configlet release will have two new release assets: configlet_4.0.0-beta.14_linux_arm64.tar.gz configlet_4.0.0-beta.14_linux_arm64.tar.gz.minisig and extracting the archive will yield the executable: $ file ./configlet ./configlet: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped Also support running `nimble build -d:zig` locally. For now, the new executable has a large comment section: $ readelf -p .comment ./configlet String dump of section '.comment': [ 0] clang version 16.0.6 (https://github.com/ziglang/zig-bootstrap 1dda86241204c4649f668d46b6a37feed707c7b4) [...] (repeat for 39 KiB) [ 8d82] clang version 16.0.6 (https://github.com/ziglang/zig-bootstrap 1dda86241204c4649f668d46b6a37feed707c7b4) [ 8deb] Linker: LLD 16.0.6 Try to strip it later. Our `strip` doesn't support elf64-aarch64 as a target, and `zig objcopy` doesn't have a `-R, --remove-section` option currently. But prepare to use `llvm-strip`. Cross-compiling for arm64 macos currently produces an error: CC: ../nimdir/lib/std/sysrand.nim /home/runner/.cache/nim/configlet_r/@m..@snimdir@slib@sstd@ssysrand.nim.c:9:10: fatal error: 'Security/SecRandom.h' file not found which is due to a recent commit [1]. See some posts on cross-compiling with Zig [2][3][4] and the support table [5]. Refs: #24 Refs: #122 Refs: #764 [1] 53a75a2 ("nimble, uuid: generate UUIDs via std/sysrand, not pragmagic/uuids", 2023-08-07) [2] https://ziglang.org/learn/overview/#cross-compiling-is-a-first-class-use-case [3] https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html [4] https://www.uber.com/blog/bootstrapping-ubers-infrastructure-on-arm64-with-zig/ [5] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
Continue the recent `zig cc` work [1], such that the next configlet release will have two new release assets: configlet_4.0.0-beta.14_macos_arm64.tar.gz configlet_4.0.0-beta.14_macos_arm64.tar.gz.minisig where the archive contains the executable: $ file ./configlet configlet: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|HAS_TLV_DESCRIPTORS> `configlet uuid` now uses [2] `std/sysrand`, which on macOS uses [3] the Security Framework. So to produce the macOS arm64 configlet, it's easiest to cross-compile from macOS x86_64, where the macOSX SDK is already available. Make the cross-compile and install-zig scripts support macOS, and add a build job for it. It looks like the macos-12 GitHub runner has SDK versions 12.3 and 13.1 installed. Use the latest one. The new executable is about 1.09 MiB, which is notably larger than the 655 KiB macOS x86_64 executable. Possible ways to reduce the size in the future: - Enable LTO when `zig ld` supports it [4]. - Compile it natively, when GitHub begins to provide a hosted arm64 macOS runner [5]. - Compile it natively, using a non-GitHub arm64 macOS runner. Refs: #24 Refs: #122 Refs: #764 [1] 0e8d665, ".github, config: use Zig to cross-compile arm64 Linux asset", 2023-08-13 [2] 53a75a2, "nimble, uuid: generate UUIDs via std/sysrand, not pragmagic/uuids", 2023-08-07 [3] https://github.com/nim-lang/Nim/blob/v2.0.0/lib/std/sysrand.nim#L235-L256 [4] https://www.github.com/ziglang/zig/issues/8680 [5] https://www.github.com/github/roadmap/issues/528
Continue the recent `zig cc` work [1], such that the next configlet release will have two new release assets: configlet_4.0.0-beta.14_linux_riscv64.tar.gz configlet_4.0.0-beta.14_linux_riscv64.tar.gz.minisig where the archive contains the executable: $ file ./configlet ./configlet: ELF 64-bit LSB executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), statically linked, stripped The riscv64-linux-musl target will have Tier 1 Zig support [2]. Refs: #24 [1] 0e8d665, ".github, config: use Zig to cross-compile arm64 Linux asset", 2023-08-13 [2] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
Continue the recent `zig cc` work [1][2][3], such that the next configlet release will have two new release assets: configlet_4.0.0-beta.14_windows_arm64.zip configlet_4.0.0-beta.14_windows_arm64.zip.minisig where the archive contains the executable: $ file ./configlet.exe ./configlet: PE32+ executable (console) Aarch64, for MS Windows, 6 sections The aarch64-windows-gnu target will have Tier 1 Zig support [4]. Also make the cross-compilation jobs and scripts work on Windows, even though we currently cross-compile the new executable from Linux. Refs: #24 Refs: #122 Closes: #764 [1] 0e8d665, 2023-08-16, ".github, config: use Zig to cross-compile arm64 Linux asset" [2] f280445, 2023-08-17, ".github: cross-compile arm64 macOS asset" [3] a962b18, 2023-08-17, ".github: cross-compile riscv64 Linux asset" [4] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
@ErikSchierboom you wrote (albeit nearly 3 years ago):
Did this mean:
And how much do we care about this? Was it just motivated by the assets we had for v2-configlet? It looks like
But I'm not keen on providing builds for them because:
Thoughts? Adding further platforms isn't a priority for me. But if I do it for my own interest, next would be x86_64 BSDs (I've built and tested configlet on them in the past) and more 64-bit Linux builds (e.g. powerpc64). |
It was, so I'm happy to ditch that. We've also hadn't really had people requesting 32-bit support AFAICT so 🤷 |
In particular, we should be supporting:
Edit by @ee7:
@ee7 is aiming towards something like:
linux_x86-64
macos_x86-64
windows_x86-64
then arm64 (AKA aarch64):
linux_arm64
(added in 0e8d665)macos_arm64
(added in f280445)windows_arm64
(added in 3824299)then:
linux_powerpc64
(which is big endian) (will be added by #799)linux_riscv64
(added in a962b18)then:
freebsd_x86-64
(if viazig cc
, blocked by ziglang/zig#2876)netbsd_x86-64
(if viazig cc
, blocked by ziglang/zig#2877)openbsd_x86-64
(if viazig cc
, blocked by ziglang/zig#2878)Edit 2023-08-22: These 32-bit builds are possible, but I think are probably not worthwhile:
linux_arm
linux_i386
linux_mipsel
linux_powerpc
windows_arm
windows_i386
see also the Zig 0.11.0 Support Table.
The text was updated successfully, but these errors were encountered: