Emit Markers when Locking

[mitsuhiko]

Description

I'm investigating pipgrip for rye (astral-sh/rye#82) and one of the topics of interest is making portable lock files that retain the markers (eg: python_version < '3.7'). It does not seem like markers are supported currently and it would appear that pipgrip can only resolve against the current Python version. I did not look into it yet, but how much work would it be to make it possible to create lock files for multiple markers?

Use case / motivation

It would be nice to have a single lock file that is Python version or platform independent.

[ddelange]

Hi @mitsuhiko 👋

The below example is run from a cp311 venv on my Mac. You can see the two relevant numpy entries in the tree, but only the final resolved version in the lock.

Main blockers from the top of my head:

• The pip subprocess can't be influenced to 'act as if this is a windows system' to download the correct wheel, or even do a source build for windows if there are no windows wheels, which can write platform specific install_requires (without markers) on setup.py runtime if the developer was feeling like it. There is also the matter of different python implementations like CPython, Pypy etc which can also be coded in the environment markers.
• The lockfiles from pipgrip are meant to be compatible with pip's -c/--constraint flag, which means that, although it can contain markers, it can only contain the == version specifier. Combine that with the fact that PubGrub doesn't have a notion of environment markers (and consequently no notion of multiple versions for a single package in the solution), I think there is no other option than to run through the dep tree multiple times (run pipgrip multiple times) from different environments, as the said environment will most likely influence the resolving decisions (drastically).

That being said: this is the environment in which the wheel METADATA is evaluated (two out of three numpy entries from below are retained here, and one is dropped). Changing/monkeypatching this is of not much use however, due to point 1 above.

Hope that helps!

$ pipgrip --tree pandas
pandas (1.5.3)
├── numpy>=1.21.0 (1.24.2)
├── numpy>=1.23.2 (1.24.2)
├── python-dateutil>=2.8.1 (2.8.2)
│   └── six>=1.5 (1.16.0)
└── pytz>=2020.1 (2022.7.1)

$ pipgrip pandas
pandas==1.5.3
numpy==1.24.2
python-dateutil==2.8.2
six==1.16.0
pytz==2022.7.1