Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plumb reproducible build env vars more thoroughly. #2554

Merged
merged 2 commits into from
Oct 9, 2024

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Oct 8, 2024

This helps reduce cache sizes where sdists are built into wheels as well
as expanding the set of sdists Pex can lock reproducibly.

This helps reduce cache sizes where sdists are built into wheels as well
as expanding the set of sdists Pex can lock reproducibly.
@jsirois jsirois force-pushed the reproducible-builds/pip-and-pep-517 branch from c3c12e1 to f173109 Compare October 9, 2024 21:39
@jsirois
Copy link
Member Author

jsirois commented Oct 9, 2024

For the record, the Pip cache size difference where the cache is blown away, then tox -epy311-integration is run before and after this change:

:; du -sh ~/.cache/pex.it-full/pip/0/ /home/jsirois/.cache/pex/pip/1/
389M    /home/jsirois/.cache/pex.it-full/pip/0/
362M    /home/jsirois/.cache/pex/pip/1/

jsirois added a commit to jsirois/unstructured that referenced this pull request Oct 9, 2024
Without going the extra length of exporting `PYTHONHASHSEED=0` or
similar, the built sdist and wheel for a given version of the project
would have inconsistent random ordering of extras_require entries
between builds.

This was observed attempting to lock a VCS requirement on this project
here: pantsbuild/pants#21145
On the bright side, it led to Pex fixing its building and locking code
to be robust to this sort of (unintended) bad behavior:
pex-tool/pex#2554
jsirois added a commit to jsirois/unstructured that referenced this pull request Oct 9, 2024
Without going the extra length of exporting `PYTHONHASHSEED=0` or
similar, the built sdist and wheel for a given version of the project
would have inconsistent random ordering of extras_require entries
between builds.

This was observed attempting to lock a VCS requirement on this project
here: pantsbuild/pants#21145
On the bright side, it led to Pex fixing its building and locking code
to be robust to this sort of (unintended) bad behavior:
pex-tool/pex#2554
@jsirois jsirois requested review from benjyw, kaos and huonw October 9, 2024 22:12
@jsirois
Copy link
Member Author

jsirois commented Oct 9, 2024

Pants folks: added you FYI since this fixed a problem observed by a Pants user here: pantsbuild/pants#21145 and so you may want this release since that user didn't actually attempt to fix the underlying problem in the wild with the "unstructured" project's build.

@jsirois jsirois merged commit cf04a5a into pex-tool:main Oct 9, 2024
23 checks passed
@jsirois jsirois deleted the reproducible-builds/pip-and-pep-517 branch October 9, 2024 23:00
@benjyw
Copy link
Collaborator

benjyw commented Oct 17, 2024

pantsbuild/pants#21545 upgrades Pants to use Pex 2.20.3, which includes this change.

benjyw added a commit to pantsbuild/pants that referenced this pull request Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants