Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

macOS wrapped_clang should export ZERO_AR_DATE=1 to make linking hermetic #10886

Closed
brentleyjones opened this issue Mar 2, 2020 · 4 comments
Closed

macOS wrapped_clang should export ZERO_AR_DATE=1 to make linking hermetic #10886

brentleyjones opened this issue Mar 2, 2020 · 4 comments
Assignees
@oquenchil
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: feature request

Comments

@brentleyjones
Copy link
Contributor

Description of the problem / feature request:

wrapped_clang, unlike bazel/libtool, doesn't export ZERO_AR_DATE=1. This causes timestamps to be embedded in linked binaries.

Feature requests: what underlying problem are you trying to solve with this feature?

wrapped_clang should export ZERO_AR_DATE=1.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

On macOS, using the Xcode toolchain:

bazel clean
bazel build :some_objc_target
cp bazel-bin/some_objc_target /tmp/some_objc_target1
bazel clean
bazel build :some_objc_target
cp bazel-bin/some_objc_target /tmp/some_objc_target2
cmp /tmp/some_objc_target1 /tmp/some_objc_target2

The binaries will report differences because of the embedded timestamps.

What operating system are you running Bazel on?

macOS

What's the output of bazel info release?

release 2.1.0

Have you found anything relevant by searching the web?

bazelbuild/rules_swift#136
bazelbuild/rules_swift#120

bazel/tools/objc/libtool.sh

Line 115 in 0e1100f

export ZERO_AR_DATE=1

Any other information, logs, or outputs that you want to share?

http://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html

The above blog mentions setting this value to remove the timestamps.
@brentleyjones
Copy link
Contributor Author

A workaround for now is to call the build with --action_env=ZERO_AR_DATE=1

@brentleyjones brentleyjones changed the title macOS wrapped_clang should set ZERO_AR_DATE=1 to make linking hermetic macOS wrapped_clang should export ZERO_AR_DATE=1 to make linking hermetic Mar 2, 2020
@irengrig irengrig added team-Rules-CPP Issues for C++ rules untriaged labels Mar 6, 2020
@oquenchil oquenchil self-assigned this Mar 13, 2020
@oquenchil oquenchil added P3 We're not considering working on this, but happy to review a PR. (No assignee) type: feature request and removed untriaged labels Mar 13, 2020
@thii
Copy link
Member

thii commented Jul 27, 2020

ZERO_AR_DATE=1 is a subset of -D, so it looks like passing -D is the preferred way now. There's a PR for it: #11576.

ZERO_AR_DATE=1 also has a bug with timezone.

The problem with this flag is that it resets the file mtime on the file to epoch=0, e.g. 1970-1-1 or 1969-12-31 depending on timezone.

thii added a commit to thii/bazel that referenced this issue Jul 27, 2020
@thii
Support deterministic mode when using a version of libtool that supports
4fd0f5a 
it.

Xcode 10.2 adds a new flag `-D` to libtool that sets archive contents'
user ids, group ids, dates, and file modes to reasonable defaults.

This patch adds this new flag to all libtool actions to when they're
built with a version of libtool that supports this mode, to ensure
hemertic results for linking actions. Note that this new flag covers the
functionality of the `ZERO_AR_DATE` environment variable, but this
variable is still being left in the libtool wrapper for
backward-compatibility with older versions.

Fixes bazelbuild#10886.

RENOTES: None.
@thii thii mentioned this issue Jul 27, 2020
Closed
thii added a commit to thii/bazel that referenced this issue Aug 14, 2020
@thii
Support deterministic mode when using a version of libtool that supports
4bca20c 
it.

Xcode 10.2 adds a new flag `-D` to libtool that sets archive contents'
user ids, group ids, dates, and file modes to reasonable defaults.

This patch adds this new flag to all libtool actions to when they're
built with a version of libtool that supports this mode, to ensure
hemertic results for linking actions. Note that this new flag covers the
functionality of the `ZERO_AR_DATE` environment variable, but this
variable is still being left in the libtool wrapper for
backward-compatibility with older versions.

Fixes bazelbuild#10886.

RENOTES: None.
@mboes mboes mentioned this issue Dec 29, 2020
Closed
@brentleyjones
Copy link
Contributor Author

This is still an issue, btw. While libtool uses -D, we need wrapped_clang to export ZERO_AR_DATE=1 to fix linking binaries (not archives).

@keith keith mentioned this issue Feb 22, 2021
Closed
@keith
Copy link
Member

keith commented Feb 22, 2021

PR to fix the link action #13091

bazel-io pushed a commit that referenced this issue Feb 26, 2021
@keith @copybara-github
Add ZERO_AR_DATE to apple_env
073ae81 
This variable disables timestamps in archives and binary links in
Apple's toolchain. Previously this was set for archives but not the
links themselves.

Relevant logic in ld64: https://github.com/keith/ld64/blob/86955a06a7e8e0d18ab28e521aa1122bfecd05a9/src/ld/Options.cpp#L4420-L4422

Fixes the other half of #10886

Closes #13091.

PiperOrigin-RevId: 359724927
@siddharthab siddharthab mentioned this issue Apr 9, 2021
Merged
siddharthab pushed a commit to grailbio/rules_r that referenced this issue Apr 10, 2021
Reproducibility of native code on macOS (#77)
e78417b 
LLVM versions on macOS have been updated to be past LLVM 7, so
-fdebug-prefix-map now works as expected and the reproducibility tests
can be re-enabled.

Now, we just need to set ZERO_AR_DATE=1 (thanks @chrchang).
Also see bazelbuild/bazel#10886

There is still a problem with the version of `install_name_tool`
shipped with Xcode CLT, and therefore we are starting to distribute a
newer version of `install_name_tool` in this project.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@oquenchil oquenchil

Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: feature request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support deterministic mode when using a version of libtool that supports it. thii/bazel
5 participants
@brentleyjones @keith @thii @irengrig @oquenchil