Pip inconsistent on whether it looks up a pre-release based on the operator of the specifier

[notatallshaw]

Description

Pip will choose a pre-release for a package if the specifier of the package includes a pre-release version.

For example pandas>=2.1.5rc01 will currently install pandas-2.2.0rc0. But if for some operators this doesn't appear to work, for example pandas>2.1.5rc01 results in ERROR: No matching distribution found for pandas>2.1.5rc01

Expected behavior

If the specifier selects a pre-release version than Pip should look for pre-releases that match that specifier regardless of the operator choice.

pip version

23.3.2

Python version

3.11

OS

Linux

How to Reproduce

1. Be on a date before Pandas 2.2.0 is installed or use pypi-timemachine to set the date to 2024-01-09
2. pip install --dry-run --no-deps --ignore-install "pandas>2.1.5rc0"
3. pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01"

Output

pandas>2.1.5rc01

pip install --dry-run --no-deps --ignore-install "pandas>2.1.5rc01"
ERROR: Could not find a version that satisfies the requirement pandas>2.1.5a1 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0rc0, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0)
ERROR: No matching distribution found for pandas>2.1.5rc01

pandas>=2.1.5rc01

pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01"
Collecting pandas>=2.1.5rc01
  Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Would install pandas-2.2.0rc0

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

The problem here is probably compounded when Pip is backtracking through transitive dependencies, in that case the user has no control over what operators are being chosen.

[notatallshaw]

While unintuitive (and potentially creating logically inconsistent situations), it appears this behavior comes from the packaging library and had a reasoning behind it: pypa/packaging#776 (comment)

It also appears to me that the packaging specification is not clear enough on pre-releases and it's ambigious whether packaging is following the spec or not.

Therefore further discussion should either happen on the packaging library or with the python packaging community: https://discuss.python.org/c/packaging