Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ability to select bytecode invalidation mode of generated .pyc #2297

Merged
merged 6 commits into from
Mar 8, 2024

Conversation

wimglenn
Copy link
Contributor

@wimglenn wimglenn commented Mar 8, 2024

Since Python 3.7, deterministic pycs are possible (see PEP 552)
To select the bytecode invalidation mode explicitly by env var:

PYC_INVALIDATION_MODE=UNCHECKED_HASH uv pip install --compile ...

Valid values are TIMESTAMP (default), CHECKED_HASH, and UNCHECKED_HASH.
The latter options are useful for reproducible builds.

wimglenn and others added 2 commits March 8, 2024 01:23
…files.

Since Python 3.7, deterministic pycs are possible (see https://peps.python.org/pep-0552/)
To select the bytecode invalidation mode explicitly by env var:

   PYC_INVALIDATION_MODE=UNCHECKED_HASH uv pip install --compile ...

Valid values are TIMESTAMP (default), CHECKED_HASH, and UNCHECKED_HASH
The latter options are useful for reproducible builds.
@konstin
Copy link
Member

konstin commented Mar 8, 2024

Thanks!

I added specific error handling and a test and fixed a bug the error handling surfaced.

$ PYC_INVALIDATION_MODE=bogus cargo run -q pip install --compile django
Resolved 3 packages in 4ms
Installed 3 packages in 45ms
error: Failed to bytecode compile /home/konsti/projects/uv/.venv/lib/python3.12/site-packages
  Caused by: Python process stderr:
Invalid value for PYC_INVALIDATION_MODE "bogus", valid are "TIMESTAMP", "CHECKED_HASH", "UNCHECKED_HASH":
  Caused by: Bytecode compilation failed, expected "/home/konsti/projects/uv/.venv/lib/python3.12/site-packages/asgiref/sync.py", received: ""

@charliermarsh and/or @zanieb: Do we have a good spot to document advanced non-cli configuration such as PYC_INVALIDATION_MODE?

@konstin konstin added the enhancement New feature or request label Mar 8, 2024
@konstin
Copy link
Member

konstin commented Mar 8, 2024

(Sorry for the CI failures, it's passing locally for me)

@zanieb
Copy link
Member

zanieb commented Mar 8, 2024

@konstin I'd just add a "Compilation of Python modules" section to the "Advanced Usage" part of the README and explain --compile and this environment variable.

@konstin
Copy link
Member

konstin commented Mar 8, 2024

Made a separate issue for: #2303

@konstin konstin merged commit 1181aa9 into astral-sh:main Mar 8, 2024
7 checks passed
@wimglenn wimglenn deleted the hash-based-invalidation branch March 8, 2024 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants