Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python 3.12, setuptools and venv #54

Merged
merged 8 commits into from
Jul 23, 2024
Merged

python 3.12, setuptools and venv #54

merged 8 commits into from
Jul 23, 2024

Conversation

elikoga
Copy link
Member

@elikoga elikoga commented Jun 17, 2024

Since python/cpython#95299 we need to ensure we have a setuptools ourselves.
They changed pip freeze behaviour in python 3.12 too: pypa/pip#12032

This leaves us with 2 options:

  1. We could lock all dependencies with pip freeze --all but:
    • this would also lock pip, wheel to a specific version, which is a change in behaviour
    • we would need to update the requirements.lock to update the version of pip used
  2. We could forbid projects with mixed python >= 3.12 and < 3.12 versions in their appenv-python-preference list
    • this would prevent the issue from happening in the first place
    • we would need to inform users about this change

To understand the second option better, it is important to know that batou only uses the first python version in the appenv-python-preference list that is actually installed on the system.

This means that if you have 3.12,3.11 as your appenv-python-preference and 3.12 is installed, batou will use 3.11 for lockfile preparation and 3.12 for venv creation. The resulting lockfile will not contain setuptools and pip freeze will not show setuptools either. This lockfile is incompatible with python 3.12, and once a user without 3.12 installed tries to create a venv, they will run into the issue described above.

Also, if you have 3.12,3.11 as your appenv-python-preference and 3.11 is installed, batou will use 3.11 for lockfile preparation and 3.11 for venv creation. The resulting lockfile will not contain setuptools and pip freeze will not show setuptools either. This does not immediately cause an issue, but once a user with 3.12 installed tries to create a venv, they will run into the issue described above.

This PR does pip freeze --all --exclude pip if both python >= 3.12 and < 3.12 are in the appenv-python-preference list. This mimics the behaviour of pip freeze after the change in python 3.12.

@elikoga
Copy link
Member Author

elikoga commented Jun 17, 2024

~~Hm this doesn't look good. It seems like the appenv environment depends on the executing python version due to the unconditional pip install pip --upgrade.

With newer python versions, creating a new venv doesn't include setuptools anymore, but batou requires setuptools and complains.~~

The correct choice is not to modify the pip freeze call, but instead install setuptools

@elikoga elikoga force-pushed the fix-pip-freeze-lockfile branch from 8edf949 to ee26e52 Compare June 20, 2024 00:38
@elikoga elikoga changed the title Update pip freeze command in appenv.py Install setuptools into venv if python >= 3.12 for backwards compatibility Jun 20, 2024
@elikoga
Copy link
Member Author

elikoga commented Jun 20, 2024

Ah, I know what we need: #1

The update command needs to migrate the project for 3.12 by reinstalling the required packages and updating the lockfile based on that. In the end, the new lockfiles include an explicit setuptools dependency. The upgrade path is also handled and no erroneous project state is introduced

@zagy zagy self-requested a review June 20, 2024 09:35
@elikoga elikoga changed the title Install setuptools into venv if python >= 3.12 for backwards compatibility python 3.12, setuptools and venv Jul 23, 2024
@elikoga elikoga force-pushed the fix-pip-freeze-lockfile branch from ee26e52 to 889fd94 Compare July 23, 2024 01:45
@zagy zagy merged commit 2aee66b into master Jul 23, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants