Reproducible builds: Buildpath embedded in binaries

[kpcyrd]

hello,

I was looking at ring from a reproducible builds point of view and I noticed that rust programs that depend on ring can not be built reproducibly with a dynamic build path.

The following screenshot is from diffoscope output on a binary that includes ring and was built twice in different directories. The binaries are almost identical, except the full path to the pre-generated assembly that is embedded into the binary. I'm not sure where this happens, so a pointer would be very appreciated.

[pietro]

Hi @kpcyrd,

Ring's build is controlled by build.rs.

pregenerate_asm_main is used to generate the asm files from perlasm, and compile then into object files for windows.

ring_build_rs_main is used to build ring, you can follow how the C and ASM code is built by going through build_c_code.

Is there any difference on non-debug builds?

[briansmith]

I was looking at ring from a reproducible builds point of view and I noticed that rust programs that depend on ring can not be built reproducibly with a dynamic build path.

I (currently) only support ring when it is used from one of the packages I build and upload to crates.io. So I think your reproducible build project reduces to "How can I reproduce the exact ring package I upload to crates.io?" In particular I don't support git = { } or forks of ring and I only support "vendoring" ring by vendoring the ring crate I produce.

How are you doing the "vendoring" of ring in your project?

[kpcyrd]

@briansmith ring is pulled into the build process by rustls, which is just pulling it from crates.io.

The problem I'm facing isn't how the ring crate is built, it's that $CARGO_HOME/registry/src/gh.neting.cc-something/ring-0.13.2/pregenerated/sha256-x86_64-elf.S is included in my final binary that depends on ring. If we can get rid of that my build process is always building an identical binary, regardless of the build path or the home directory.

I'm using --remap-path-prefix= on both $HOME and $PWD which should normalize that but it doesn't catch this one.

@pietro could you give me instructions how to do that? :)

[pietro]

@kpcyrd I meant doing cargo build --release instead of just cargo build. We should look into conditionally passing -fdebug-prefix-map=$(pwd)=. to GCC and Clang. Can you share with me scxripts you are using to build and test rustls with diffoscope? Thanks.

[pietro]

Yep, -fdebug-prefix-map=$(pwd)=. fixes this. https://gist.github.com/pietro/cd1ac0ad985dc9e163d752bd6bbd92d7 is a quick and dirty way to pass the flag to the compiler using a feature.

[pietro]

export CFLAGS="-fdebug-prefix-map=$(pwd)=." works without any modification to the source code.

[briansmith]

export CFLAGS="-fdebug-prefix-map=$(pwd)=." works without any modification to the source code.

Is there any reason to not just do this in build.rs by default?

[pietro]

It looks like the option is supported in GCC since 4.3 and in clang since 3.8. I'll look into adding it to the non msvc flags.

[kpcyrd]

I found a work-around in the meantime:

#!/bin/sh
CARGO_HOME="${CARGO_HOME:-$HOME/.cargo}"
mkdir -p -- "$CARGO_HOME"
unshare -Umr sh -xe <<EOF
mount -t tmpfs tmpfs /mnt
mkdir /mnt/src /mnt/cargo
mount --bind "$PWD" /mnt/src
mount --bind "$CARGO_HOME" /mnt/cargo
cd /mnt/src/
CARGO_HOME=/mnt/cargo cargo build --release --verbose --target=x86_64-unknown-linux-musl
EOF

This sets up a user namespace to ensure the embedded build paths are always going to be /mnt/cargo and /mnt/src, regardless of the directory the build is originally started in.

[briansmith]

There are two open PRs I know of regarding reproducible builds: PR #802 and now PR #1697.

I have written PR #1697 to clarify how the build configuration is done, and to make it easier to land the change in PR #802.

[briansmith]

I'm happy to take a PR to fix this in build.rs. Basically somebody needs to do what PR #802 does, but rebased on main to resolve conflicts and use OsStr/OsString instead.

[TheButlah]

Hi, this bit me today. Looking at #802, it looks like a simple fix. Is the above guidance for what a contributor would need to do to get it merged still accurate?