pip download: python_version in requirements file and --python-version option does not work

[sashkab]

Description

Consider following requirements file:

$ cat requirements.txt
numexpr==2.7.3;python_version=="3.8"
numexpr==2.8.1;python_version=="3.9"

Then running following command with pip using python 3.10:

python3.10 -mpip download --only-binary=":all:" --no-cache-dir \
   --disable-pip-version-check --platform=linux_x86_64 \
   --python-version=3.8 --implementation=cp --prefer-binary \
   --no-index -d dst -f src -r requirements.txt

All wheels are available in the src directory.

Unfortunately, download fails with following error:

Ignoring numexpr: markers 'python_version == "3.8"' don't match your environment
Ignoring numexpr: markers 'python_version == "3.9"' don't match your environment

Expected behavior

wheel for numexpr 2.7.3 would be downloaded.

pip version

22.1.2

Python version

3.10.4

OS

macOS, linux

How to Reproduce

1. Create requirements.txt file:

   numexpr==2.7.3;python_version=="3.8"
   numexpr==2.8.1;python_version=="3.9"
   

2. Ensure wheels are available in the src directory.

3. Run command

   python3.10 -mpip download --only-binary=":all:" --no-cache-dir --disable-pip-version-check \
     --platform=linux_x86_64 --python-version=3.8 --implementation=cp \
     --prefer-binary --no-index -d dst -f src -r requirements.txt
   

Output

$ python3.10 -mpip download --only-binary=":all:" --no-cache-dir --disable-pip-version-check \
  --platform=linux_x86_64 --python-version=3.8 \
  --implementation=cp --prefer-binary --no-index -d dst -f src -r requirements.txt
Ignoring numexpr: markers 'python_version == "3.8"' don't match your environment
Ignoring numexpr: markers 'python_version == "3.9"' don't match your environment

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

@sashkab Thanks for filing this issue.

wheel for numexpr 2.7.3 would be downloaded.

Why do you expect this? When using Python 3.10, python_version is equal to "3.10" and "3.8" != "3.10". So, both of those markers evaluate to False.

[pradyunsg]

If you add numexpr==2.7.3;python_version=="3.10", you will get the expected behaviour.

[pfmoore]

@pradyunsg I assume the OP is expecting --python-version to affect the evaluation context of the markers. (Edit: I notice you need to scroll the command line to see that option was specified...)

[sashkab]

@pradyunsg I edited description to show that I supply --python-version=3.8 option to the PIP. My understanding of the option is that this option would overwrite running interpreter's version, according to the help:

  --python-version <python_version>
                              The Python interpreter version to use for wheel and "Requires-Python" compatibility checks. Defaults to a version derived from the running
                              interpreter. The version can be specified using up to three dot-separated integers (e.g. "3" for 3.0.0, "3.7" for 3.7.0, or "3.7.3"). A major-
                              minor version can also be given as a string without dots (e.g. "37" for 3.7.0).

Did I misunderstood the explanation?

[pradyunsg]

Ah, right. --python-version only affects which wheel is picked from package indexes, but does not affect marker evaluation at this time.

[sashkab]

@pradyunsg what would be the workaround at this point? My use-case – if I'm preparing docker-image and have access only to python3.10, but need to download wheels for other python versions?

[sbidoul]

As far as I know there is now workaround at the moment. I'm marking this as awaiting PR.

At first sight, one needs to pass options such as --python-version, --platform, --implementation to the resolver (in addition to the finder) under the form of some "environment override" so they can be used in markers.evaluate(). One has to investigate if the format of these options is compatible with PEP 508 environment markers.

[sbidoul]

Actually I think we can close this as a duplicate of #10050 which as more context and discussion of a possible approach towards a solution.