Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: don't require system Python to perform bootstrapping #1929

Merged
merged 2 commits into from
Jun 2, 2024
Merged

fix: don't require system Python to perform bootstrapping #1929

merged 2 commits into from
Jun 2, 2024
+1,440 −81

Commits on Jun 2, 2024

  1. fix: don't require system Python to perform bootstrapping 

    This is a pretty major, but surprisingly non-invasive, overhaul of how
binaries are started. It fixes several issues and lays ground work for
future improvements.

In brief:

* A system Python is no longer needed to perform bootstrapping.
* Errors due to `PYTHONPATH` exceeding environment variable size limits
  is no longer an issue.
* Coverage integration is now cleaner and more direct.
* The zipapp `__main__.py` entry point generation is separate from the
  Bazel binary bootstrap generation.
* Self-executable zips now have actual bootstrap logic.

The way all of this is accomplished is using a two stage bootstrap
process. The first stage is responsible for locating the interpreter,
and the second stage is responsible for configuring the runtime
environment (e.g. import paths). This allows the first stage to be
relatively simple (basically find a file in runfiles), so implementing
it in cross-platform shell is feasible. The second stage, because it's
running under the desired interpreter, can then do things like setting
up import paths, and use the `runpy` module to call the program's real
main.

This also fixes the issue of long `PYTHONPATH` environment variables
causing an error. Instead of passing the import paths using an
environment variable, they are embedded into the second stage bootstrap,
which can then add them to sys.path.

This also switches from running coverage as a subprocess to using its
APIs directly. This is possible because of the second stage bootstrap,
which can rely on `import coverage` occurring in the correct
environment.

This new bootstrap method is disabled by default. It can be enabled
by setting
`--@rules_python//python/config_settings:bootstrap_impl=two_stage`.
Once the new APIs are released, a subsequent release will make it the
default. This is to allow easier upgrades for people defining their
own toolchains.
    @rickeylev
    rickeylev committed Jun 2, 2024
    Configuration menu
    Copy the full SHA
    b650709 View commit details
    Browse the repository at this point in the history

  2. default to sys py; remove todo

    @rickeylev
    rickeylev committed Jun 2, 2024
    Configuration menu
    Copy the full SHA
    b7b7b3f View commit details
    Browse the repository at this point in the history