sign tarballs with PGP instead of using a sha256 checksum

[MageJohn]

I'm using Manjaro, which is Arch Linux based and has access to the AUR, so I'm getting Zig by using the zig-dev-static package. However I can't use the one at that link directly because it can be out of date within hours. I download the script and manually update the version and sha256 sums in it. Well, actually I have a bash script that uses the JSON API to do it for me. But I had to write that manually. The point is there are a couple of things that would be nice to have which would make packaging easier:

• A consistent link for the latest build. Something like https://ziglang.org/builds/zig-linux-x86_64-latest.tar.xz. It's not possible to get makepkg to somehow use the JSON API.
• A PGP signature for the build. Again, the SHA sum from the JSON API can't be used, but a signature file can be added to the sources array, and will be used to automatically check the file.
• A monotonically increasing version number. Right now the latest version is often registered as a downgrade, because of the commit id. The easiest way is to use git describe --long, or even better follow the recommendation on the Arch Wiki: https://wiki.archlinux.org/index.php/VCS_package_guidelines#The_pkgver()_function. The version number can be updated with a function in the PKGBUILD (only after the sources have been downloaded), so the JSON API can be used.

These changes would allow installing the latest Zig build directly from the AUR with zero friction. And hopefully they're helpful to more than just AUR users as well!

[andrewrk]

The other two are possible but unfortunately the consistent link for the latest build is tricky due to limitations of s3, the (current) backend of ziglang.org. We used to have this but it used permanent redirects instead of temporary, and s3 does not have a way to configure it. It ended up causing more problems than it solved.

[ikskuh]

I have written a zig-up script that will update a locally installed zig version to the latest build from the website, it depends on bash, tar and jq, script is tested and used on an archlinux, but needs adjustments to other systems (and maybe user names grin)

As zig has no external dependencies, it should be enough to just dispatch this script (or even better: rewrite it in zig!) to provide the current master build of zig.

[MageJohn]

Some sort of zig-up tool would be useful, indeed. Especially if it also helps manage multiple versions of Zig like rustup does. The language is evolving fast, and sometimes introduces incompatibilities that might mean that someone has projects on the same machine requiring different versions of Zig. This may not be the issue to discuss this though.

[andrewrk]

• A consistent link for the latest build.

Sadly impossible due to using S3 for ziglang.org.

• A PGP signature for the build.

Planned starting with the upcoming 0.9.0 release. I'll keep this issue open to track it.

• A monotonically increasing version number.

Done since #6509.

Some sort of zig-up tool would be useful, indeed

There are existing community solutions to this, such as zigup. See also #10204.

[andrewrk]

Alright I still want to do this but I'm not going to try to rush it 2 weeks before the 0.9.0 release. Will take my time setting this up and do it as part of the 0.10.0 release cycle.

[jedisct1]

Minisign is also an option for signing builds, especially since there's a pure Zig version that can be used for verification.

Minisign signatures are also supported by the Sigstore transparency log, which is an important piece of the puzzle for supply chain security.

[Techcable]

This is a problem for me because it makes it tough for users to reliably download as part of a AUR-style build system.

The AUR package for zig-dev-bin is in a tricky situation:

Generally speaking, AUR packages are expected to either GPG or SHA256 verify the files they are downloading.

Right now now GPG verification is not possible (due to the issue). Ideally the package would auto-updating to the latest hightly build, but that would make hardcoding sha256 checksums impossible.

The sha256 in version-index.json is not helpful for my purposes. It just moves the integrity problem into a different file.

Right now the current approach the maintainer makes is to pin against specific nightly versions and use a hardcoded SHA256 hash. He updates it every couple of weeks. This is obviously not a great approach (it requires manual intervention by maintainer, and uses outdated builds), but the alternative would be "unverified" files, relying only on the integrity of the https network connection :(

I have a patch to make the aur package auto-update, but I'm not sure the maintainer will accept it for this integrity reason. It might even be against AUR policy.

I know this complaining isn't particularly helpful, but I thought I'd at least bring up my use-case :)

I think someone with access to the build infrastructure would have to be the one who ultimately sets this up.
Also it's fine if the GPG key is a dummy value (like "build.bot@ziglang.org",). There is no need to use a personal/organization GPG key, it's just important a signature exists 😉

Minisign is also an option for signing builds, especially since there's a pure Zig version that can be used for verification.

Minisign signatures

Not sure that this would fit my use case (or AUR policy). Unfortunately, minisign is not quite widely accepted yet

[maxmilton]

I'm on Arch Linux and ran into the problem of the outdated AUR package as well.

Rather than wait for this, I've created my own small bash script that also verifies the shasum hash and file size. It's based on the gist shared before, thanks for that @MasterQ32 !

[Techcable]

Rather than wait for this, I've created my own small bash script that also verifies the shasum hash and file size. It's based on the gist shared before, thanks for that @MasterQ32 !

Really, this does not solve the underlying security problem. If someone can intercept your HTTPS connection to the binary they can also intercept your connection to index.json. It's the exact same domain name and similar url.

You are implicitly trusting that the HTTPS connection to https://ziglang.org/download/index.json is valid.
If someone was able to forge a trusted TLS certificate, someone could hijact your connection and present you with invalperspectiveid (although very difficult to do, it has happened before.

Checking the sha256sum does protect against honest mistakes or download errors (which happen a good deal), but downloading the checksum from the same server as the binary does not provide any integrity checks from a security .

At this point you're probably thinking "this is crazy" and "why would anyone want to intercept my HTTPS connection". I agree. I don't personally worry about anyone intercepting my HTTPS connections.

However, I am under the impression that PGP signing nightly binaries is a security best-practice. It is certainly strongly suggested on the AUR. I think we should move in that direction

This is what Rust does (even for their nightly builds). Here is a link to the latest nightly binary and ts corresponding GPG signature:
1.https://static.rust-lang.org/dist/rust-nightly-aarch64-unknown-linux-musl.tar.gz
2. https://static.rust-lang.org/dist/rust-nightly-aarch64-unknown-linux-musl.tar.gz.asc (small text file, safe to click)

These are both signed by the rust signing key

[maxmilton]

@Techcable that makes sense to me, especially given I fetch the shasum dynamically. Thank you for explaining your thinking!

A (dynamically fetched) checksum to validate the downloaded file integrity is nice, but I agree with you, tightly controlled PGP key/s sounds like a very good idea for security 👍

[andrewrk]

@jedisct1 I'm looking into setting this up. Do you recommend minisign? If so, I'll move forward with it.

Edit: I just noticed the home page and realized what it is. Looks perfect 😁

[jedisct1]

Minisign has quite a few implementations including in Zig (for verification only; I can quickly add signing).

It's supported by Google's transparency log https://docs.sigstore.dev/rekor/sign-upload/

An alternative would be to simply use OpenSSH that can now be used to sign arbitrary files: https://www.agwa.name/blog/post/ssh_signatures

[andrewrk]

How can I make minisign not prompt for an (empty) password?

Edit: looks like echo | minisign ... works. But that's going to be problematic in a bash script, if the minisign binary returns an error code.

[andrewrk]

Done in ziglang/www.ziglang.org@1164e10. Master branch signatures will arrive with the next run of the build-tarballs script.

[jedisct1]

How can I make minisign not prompt for an (empty) password?

Edit: looks like echo | minisign ... works. But that's going to be problematic in a bash script, if the minisign binary returns an error code.

echo | minisign or minisign < passwd.txt, but I can add a flag to not prompt for a password. I think the Rust implementation has that feature, so let's do it in the C version as well.

[andrewrk]

The first run of tarballs script failed because I missed a line; fixed in ziglang/www.ziglang.org@015540f. Fix should arrive on the website in about 9 hours.

[jedisct1]

minisign 0.11 is going to be released now. [UPDATE: it's released]

The -W flag skips key encryption; however you'll need to create a new key pair with that flag set.

[mitchellh]

I want to propose that we keep the SHA hashes AND sign the archives. The reason being is that various package managers (Nix, I've been told AUR) require a SHA256 sum ahead of time to download anything. So, with my Nix Zig packages, I'm now forced to download the releases to compute the sum, store that in the package descriptor, AND verify signatures. (See mitchellh/zig-overlay#13)

My usual approach I've taken with this for my software is to ship a SHA256SUMs file alongside all of my releases, and PGP sign that as well. So you end up with basically the following:

• zig-1234.tar.xz
• zig-1234.tar.xz.minisig
• zig-1234-SHA256SUMS (probably the sums for every platform here)
• zig-1234-SHA256SUMS.minisig

For Zig, perhaps the most helpful would be to provide index.json and index.json.minisig, and have the checksums be in the JSON file. Automatically generated package updaters have to get a SHA256SUMs-esque file from somewhere anyways (or compute it themselves), and downloading the one JSON file + one small sig is a lot easier to setup the metadata in a package.

The package itself could download the binary and the binary signature and verify those as part of install.

TLDR: Re-add the checksums, sign the index.json file, and some package management pain goes away.

Not something that's terribly important to me but figured I'd share this experience report.

[nektro]

is there any prior art in anyone making a web server that generates these on the fly?

[ikskuh]

is there any prior art in anyone making a web server that generates these on the fly?

That would be a horrible waste of resources, considering the tradeoff {storing 16 byte} vs {computing 16 byte from 100 MB each request}

[nektro]

zig-windows-x86_64.tar.xz is almost double the size of the next largest tarball at 71MB and the sha256 can be calculated in only .03s

and caching can be done very easily if cpu became an issue

[andrewrk]

Thanks for the feedback @mitchellh. I agree with your conclusions and I will make it so.

[andrewrk]

Done in ziglang/www.ziglang.org@af9d69d. Checksums for master branch will arrive with the next CI run. The signature of the index.json file itself is located at https://ziglang.org/builds/zig-$VERSION-index.json.minisig. Unfortunately, making it https://ziglang.org/download/index.json.minisig would be a race condition.

[mitchellh]

Thank you, that made things very easy. I now verify the signature of the index.json then trust the hashes in the manifest: mitchellh/zig-overlay@dccddf6