Make Maven build reproducible

[Marcono1234]

Purpose

Makes the Maven build reproducible

Description

Makes the Maven build reproducible, see https://maven.apache.org/guides/mini/guide-reproducible-builds.html, and also google/guava#6322. However, it might still be necessary to use the same OS and JDK version to actually be able to reproduce a build.

The changes also format the pom.xml files in the way the Maven Release Plugin would.

Checklist

• New code follows the Google Java Style Guide
  This is automatically checked by mvn verify, but can also be checked on its own using mvn spotless:check.
  Style violations can be fixed using mvn spotless:apply; this can be done in a separate commit to verify that it did not cause undesired changes.
• If necessary, new public API validates arguments, for example rejects null
• New public API has Javadoc
  • Javadoc uses @since $next-version$
    ($next-version$ is a special placeholder which is automatically replaced during release)
• If necessary, new unit tests have been added
  • Assertions in unit tests use Truth, see existing tests
  • No JUnit 3 features are used (such as extending class TestCase)
  • If this pull request fixes a bug, a new test was added for a situation which failed previously and is now fixed
• mvn clean verify javadoc:jar passes without errors

[Marcono1234]

I have tested the release workflow locally and it seems to work as expected, but I excluded the GPG plugin for my tests because I don't have it set up locally.

The output of mvn help:effective-pom -Prelease suggests it will be run after the buildinfo goal of the maven-artifact-plugin, as desired. And if I understand it correctly the GPG plugin does not modify the JARs but just attaches additional .asc signature files, so this should not be a problem (I hope).

[Marcono1234]

Have included this as separate job (instead of including it in the steps of the build above) because it requires running mvn ... install which is normally not needed for a regular build, and also to keep it independent from the 'real' build setup above.

However, this makes the GitHub workflow slightly slower though.

[Marcono1234]

This setup is a bit hacky and I am not sure if there is a proper better alternative to this, therefore I added this "Warning". But it does allow you to test the behavior of the release workflow quite accurately, which is why I added it.

[Marcono1234]

This includes information about the build environment to make it easier to reproduce the build. I am not sure if you would consider some of this information sensitive (e.g. OS name and version or Java vendor and version).

For example for me the generated file looks like this:

gson-2.10.2.buildinfo (click to expand)

# https://reproducible-builds.org/docs/jvm/
buildinfo.version=1.0-SNAPSHOT

name=Gson Parent
group-id=com.google.code.gson
artifact-id=gson-parent
version=2.10.2

# source information
source.scm.uri=scm:git:https://github.com/google/gson.git
source.scm.tag=gson-parent-2.10.2

# build instructions
build-tool=mvn

# effective build environment information
java.version=17.0.6
java.vendor=Eclipse Adoptium
os.name=Windows 10

# Maven rebuild instructions and effective environment
mvn.version=3.9.1
mvn.aggregate.artifact-id=gson

# aggregated output

outputs.0.coordinates=com.google.code.gson:gson-parent

outputs.0.0.groupId=com.google.code.gson
outputs.0.0.filename=gson-parent-2.10.2.pom
outputs.0.0.length=20999
outputs.0.0.checksums.sha512=36a916aea7d2e261e62a9287d0d9a2fcf9cdd94f1658bfba616dc3e9a08121b3ff080e5b574f32102311b8c6d286db977dd5b7038c5b4841c0fd63a164fecd93

outputs.1.coordinates=com.google.code.gson:gson

outputs.1.0.groupId=com.google.code.gson
outputs.1.0.filename=gson-2.10.2.pom
outputs.1.0.length=11602
outputs.1.0.checksums.sha512=e16df3d3a571fee3d6145007eb7eb20dcef0a7c9707d7ea347341cf3708ea73f197550076336dc3bb29c842ee4d00ad2ce5a9ae10302d98ac55d4437d3d7949c

outputs.1.1.groupId=com.google.code.gson
outputs.1.1.filename=gson-2.10.2.jar
outputs.1.1.length=307320
outputs.1.1.checksums.sha512=300dc667e042cb8dcc65fcf97b169b2b15bec96d4728164649f2e9dff2cc1a8b4b1f1053f12875fe59395fb42bac8abed49a5d410ecfcae8ca2287f9f8e9ada1

outputs.1.2.groupId=com.google.code.gson
outputs.1.2.filename=gson-2.10.2-sources.jar
outputs.1.2.length=202441
outputs.1.2.checksums.sha512=aa94a042ee8e3a985329eb421853668a2660ef5f1b70c5fdbcfa0b6f2cbb203ae07275ccaba02fb418773fc89ba36984e1671ee832ebe8eb31c97436e84761c9

[Marcono1234]

This causes a warning during the build:

[WARNING] SCM source tag in buildinfo source.scm.tag=HEAD does not permit rebuilders reproducible source checkout

But I guess that can be safely ignored because during release source.scm.tag will be temporarily changed by the Maven Release Plugin.

[eamonnmcmanus]

This all looks great. Thanks!

I'm not too concerned about including the build environment information. It will likely be Google's internal version of Linux in actual releases, which means nobody outside will be able to reproduce it exactly. But I don't think there's anything secret there.

We haven't had a release since January and there has been quite a lot of activity since then, so we will probably have the opportunity to put this in action soon.

Let me know if this is ready to merge or if you still have changes in mind.

[Marcono1234]

It will likely be Google's internal version of Linux in actual releases, which means nobody outside will be able to reproduce it exactly. But I don't think there's anything secret there.

Ok thanks for the confirmation; I wasn't completely sure how you performed the previous releases since you were shown as Git committer. Even if it is an internal version of Linux, I assume it might still be reproducible. The main concern the Maven guide listed was line endings (\n for Linux, \r\n for Windows) which could have an effect; there should hopefully not be any relevant differences between Linux versions.

We haven't had a release since January and there has been quite a lot of activity since then, so we will probably have the opportunity to put this in action soon.

I was thinking about the same some weeks ago, but I think it would be good to include a few more changes:

• Adds support to BigDecimal in JsonPrimitive#equals #2364
• Support serializing anonymous and local class with custom adapter #2498 (I will try to resolve the conflicts in the next days)
• Some minor code clean-up changes I am currently working on, but haven't created a PR yet
• The nested JSON handling we have been writing about; I will have to resolve conflicts there as well

Let me know if this is ready to merge or if you still have changes in mind.

I think this should be ready; my main concern is the GPG plugin, but in the worst case it will fail on release and we can revert the changes of this PR again temporarily.