Non-zero exit code for ERROR: pip's dependency resolver does not currently take into account all the packages that are installed?

[pilosus]

Description

I've got two packages:

• package A (with transitive dependency B<N) in requirements file first.txt
• package C (with transitive dependency B>N) in requirements file second.txt

So basically the transitive dependencies for both A and C are the same package but of conflicting versions.

If I do the following:

• pip install --no-cache-dir -r first.txt
• pip install --no-cache-dir -r second.txt

I do:

• see an error ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. in stderr when installing second.txt
• BUT despite the error pip install successfully installs conflicting transitive dependency B>N
• pip install also returns exit code 0

Expected behavior

pip install --no-cache-dir -r second.txt should throw ERROR: ResolutionImpossible and returns exit code 1 (or whatever non-zero code it should return).

Another (less attractive than the previous one) option would be just returning non-zero exit code for ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. Because it's an error, not a warning.

pip version

21.2.4

Python version

3.9.6

OS

Ubuntu 21.04, Alpine 3.14

How to Reproduce

My setup is a virtualenv with the following versions:

(pip-separate-no-cache-dir) ➜  pip-bug python --version
Python 3.9.6

(pip-separate-no-cache-dir) ➜  pip-bug pip --version
pip 21.2.4 from /home/vitaly/.pyenv/versions/3.9.6/envs/pip-separate-no-cache-dir/lib/python3.9/site-packages/pip (python 3.9)

The content of the requirement files:

(pip-separate-no-cache-dir) ➜  pip-bug cat first.txt
aiohttp-apispec==2.2.1  # webargs<6

(pip-separate-no-cache-dir) ➜  pip-bug cat second.txt
webargs-sanic==2.0.0    # webargs >=7.0.1

Steps to reproduce:

1. pip install --no-cache-dir -r first.txt
2. pip install --no-cache-dir -r second.txt
3. echo $?
4. pip check

Output

Installing the first requirement file:


(pip-separate-no-cache-dir) ➜  pip-bug pip install --no-cache-dir -r first.txt
Collecting aiohttp-apispec==2.2.1
  Downloading aiohttp-apispec-2.2.1.tar.gz (2.3 MB)
     |████████████████████████████████| 2.3 MB 2.5 MB/s
Collecting aiohttp<4.0,>=3.0.1
  Downloading aiohttp-3.7.4.post0-cp39-cp39-manylinux2014_x86_64.whl (1.4 MB)
     |████████████████████████████████| 1.4 MB 13.7 MB/s
Collecting apispec<4.0,>=3.0.0
  Downloading apispec-3.3.2-py2.py3-none-any.whl (27 kB)
Collecting webargs<6.0
  Downloading webargs-5.5.3-py3-none-any.whl (29 kB)
Collecting jinja2<3.0
  Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
     |████████████████████████████████| 125 kB 11.8 MB/s
Collecting multidict<7.0,>=4.5
  Downloading multidict-5.1.0-cp39-cp39-manylinux2014_x86_64.whl (151 kB)
     |████████████████████████████████| 151 kB 10.5 MB/s
Collecting async-timeout<4.0,>=3.0
  Downloading async_timeout-3.0.1-py3-none-any.whl (8.2 kB)
Collecting typing-extensions>=3.6.5
  Downloading typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)
Collecting attrs>=17.3.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
     |████████████████████████████████| 53 kB 12.8 MB/s
Collecting yarl<2.0,>=1.0
  Downloading yarl-1.6.3-cp39-cp39-manylinux2014_x86_64.whl (315 kB)
     |████████████████████████████████| 315 kB 11.5 MB/s
Collecting chardet<5.0,>=2.0
  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
     |████████████████████████████████| 178 kB 11.2 MB/s
Collecting MarkupSafe>=0.23
  Downloading MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)
Collecting marshmallow>=2.15.2
  Downloading marshmallow-3.13.0-py2.py3-none-any.whl (47 kB)
     |████████████████████████████████| 47 kB 19.4 MB/s
Collecting idna>=2.0
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 17.1 MB/s
Using legacy 'setup.py install' for aiohttp-apispec, since package 'wheel' is not installed.
Installing collected packages: multidict, idna, yarl, typing-extensions, marshmallow, MarkupSafe, chardet, attrs, async-timeout, webargs, jinja2, apispec, aiohttp, aiohttp-apispec
    Running setup.py install for aiohttp-apispec ... done
Successfully installed MarkupSafe-2.0.1 aiohttp-3.7.4.post0 aiohttp-apispec-2.2.1 apispec-3.3.2 async-timeout-3.0.1 attrs-21.2.0 chardet-4.0.0 idna-3.2 jinja2-2.11.3 marshmallow-3.13.0 multidict-5.1.0 typing-extensions-3.10.0.0 webargs-5.5.3 yarl-1.6.3

(pip-separate-no-cache-dir) ➜  pip-bug echo $?
0

(pip-separate-no-cache-dir) ➜  pip-bug pip check
No broken requirements found.

(pip-separate-no-cache-dir) ➜  pip-bug

Installing the second requirement file

(pip-separate-no-cache-dir) ➜  pip-bug pip install --no-cache-dir -r second.txt
Collecting webargs-sanic==2.0.0
  Downloading webargs-sanic-2.0.0.tar.gz (5.8 kB)
Collecting sanic>=0.8.3
  Downloading sanic-21.6.2-py3-none-any.whl (94 kB)
     |████████████████████████████████| 94 kB 2.4 MB/s
Collecting webargs>=7.0.1
  Downloading webargs-8.0.1-py3-none-any.whl (30 kB)
Collecting httptools>=0.0.10
  Downloading httptools-0.3.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (428 kB)
     |████████████████████████████████| 428 kB 5.5 MB/s
Collecting aiofiles>=0.6.0
  Downloading aiofiles-0.7.0-py3-none-any.whl (13 kB)
Collecting sanic-routing~=0.7
  Downloading sanic_routing-0.7.1-py3-none-any.whl (23 kB)
Collecting websockets>=9.0
  Downloading websockets-9.1-cp39-cp39-manylinux2010_x86_64.whl (102 kB)
     |████████████████████████████████| 102 kB 544 kB/s
Collecting uvloop>=0.5.3
  Downloading uvloop-0.16.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.3 MB)
     |████████████████████████████████| 4.3 MB 8.1 MB/s
Collecting ujson>=1.35
  Downloading ujson-4.1.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (179 kB)
     |████████████████████████████████| 179 kB 11.0 MB/s
Requirement already satisfied: multidict<6.0,>=5.0 in /home/vitaly/.pyenv/versions/3.9.6/envs/pip-separate-no-cache-dir/lib/python3.9/site-packages (from sanic>=0.8.3->webargs-sanic==2.0.0->-r second.txt (line 1)) (5.1.0)
Requirement already satisfied: marshmallow>=3.0.0 in /home/vitaly/.pyenv/versions/3.9.6/envs/pip-separate-no-cache-dir/lib/python3.9/site-packages (from webargs>=7.0.1->webargs-sanic==2.0.0->-r second.txt (line 1)) (3.13.0)
Using legacy 'setup.py install' for webargs-sanic, since package 'wheel' is not installed.
Installing collected packages: websockets, uvloop, ujson, sanic-routing, httptools, aiofiles, webargs, sanic, webargs-sanic
  Attempting uninstall: webargs
    Found existing installation: webargs 5.5.3
    Uninstalling webargs-5.5.3:
      Successfully uninstalled webargs-5.5.3
    Running setup.py install for webargs-sanic ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
aiohttp-apispec 2.2.1 requires webargs<6.0, but you have webargs 8.0.1 which is incompatible.
Successfully installed aiofiles-0.7.0 httptools-0.3.0 sanic-21.6.2 sanic-routing-0.7.1 ujson-4.1.0 uvloop-0.16.0 webargs-8.0.1 webargs-sanic-2.0.0 websockets-9.1

(pip-separate-no-cache-dir) ➜  pip-bug echo $?
0

# with or without --no-cache-dir option, pip check detects the problem
(pip-separate-no-cache-dir) ➜  pip-bug pip check
aiohttp-apispec 2.2.1 has requirement webargs<6.0, but you have webargs 8.0.1.

All in all, the ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. is thrown to stderr, but exit code is 0 and installation is successful.

I do understand that error says precisely that the pip couldn't properly resolve as not all package versions being taken into consideration. But still it is not a warning, but an error, so why exit code is 0?

Also, pip check does detect the problem after the installation.

By the way, for some cases if no --no-cache-dir option is used, pip throws proper ERROR: ResolutionImpossible, although I cannot reproduce the behaviour for the current exemple requirement files.

If both requirements combined into one, then resolution confict is detected correctly, non-zero exit code is returned as expected:

(pip-single) ➜  pip-bug cat single.txt
aiohttp-apispec==2.2.1  # webargs<6
webargs-sanic==2.0.0    # webargs >=7.0.1

(pip-single) ➜  pip-bug pip install --no-cache-dir -r single.txt
Collecting aiohttp-apispec==2.2.1
  Downloading aiohttp-apispec-2.2.1.tar.gz (2.3 MB)
     |████████████████████████████████| 2.3 MB 1.9 MB/s
Collecting webargs-sanic==2.0.0
  Downloading webargs-sanic-2.0.0.tar.gz (5.8 kB)
Collecting aiohttp<4.0,>=3.0.1
  Downloading aiohttp-3.7.4.post0-cp39-cp39-manylinux2014_x86_64.whl (1.4 MB)
     |████████████████████████████████| 1.4 MB 13.5 MB/s
Collecting apispec<4.0,>=3.0.0
  Downloading apispec-3.3.2-py2.py3-none-any.whl (27 kB)
Collecting webargs<6.0
  Downloading webargs-5.5.3-py3-none-any.whl (29 kB)
Collecting jinja2<3.0
  Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
     |████████████████████████████████| 125 kB 11.0 MB/s
Collecting sanic>=0.8.3
  Downloading sanic-21.6.2-py3-none-any.whl (94 kB)
     |████████████████████████████████| 94 kB 13.9 MB/s
INFO: pip is looking at multiple versions of aiohttp-apispec to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r single.txt (line 1) and -r single.txt (line 2) because these package versions have conflicting dependencies.

The conflict is caused by:
    aiohttp-apispec 2.2.1 depends on webargs<6.0
    webargs-sanic 2.0.0 depends on webargs>=7.0.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
(pip-single) ➜  pip-bug echo $?
1
(pip-single) ➜  pip-bug pip check
No broken requirements found.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

Unfortunately there are way too many people depending on pip returning 0 in this case, we can’t just change it. The return code predates the message by a loooooong time.

[pilosus]

Now that people use Docker extensively and relying on Docker layers caching heavily, installing dependencies in multiple steps is normal. But installing broken transitive dependencies silently is probably not normal.

I do understand that pip is complex, widely used, so that supporting backward compatibility is super important and changing current behaviour is not always possible. But what if an optional flag could be introduced to alter the behaviour I've reported?
E.g. --force-check to forcefully verify deps compatibility?
Or maybe --always-fail-on-errors to exit with non-zero code if any error occur?

[pilosus]

Also, maybe stressing out in pip's official docs that installing pip install x y may be very different from pip install x && pip install y would be helpful? The only place where I've managed to find something related was in the Changes to the pip dependency resolver in 20.3 (2020) section:

[...] when you run a pip install command, pip only considers the packages you are installing in that command and may break already-installed packages. It will not guarantee that your environment will be consistent all the time.

P.S. Sorry my ignorance. May be the things I am asking/suggesting is known for ages, but I've just couldn't find them before reporting. Also, bumping into the reported problem after years of Python devepoment was kind of surprising, showing that I don't really understand how pip deps resolution works internally

[pradyunsg]

As @uranusjr noted, the contract pip has had for many years is that pip install returns 0 if it installed the request. That isn't affected by this warning (an intentional design choice I made in #5000) to avoid breaking that contract. With the new resolver, we added an (IMO) better error message for this specific case, providing more context on what's happening in this specific situation.

E.g. --force-check to forcefully verify deps compatibility?

This functionality is already exposed -- it's the pip check command. Adding a && pip check at the end of that pip install command in your Dockerfile should give you the behaviour you're looking for.

[pilosus]

thanks for the clarifications!

This functionality is already exposed -- it's the pip check command. Adding a && pip check at the end of that pip install command in your Dockerfile should give you the behaviour you're looking for.

This is exactly what I've ended up with.
By proposing the flag I meant more of a way to raise awareness. It's not obvious that an error detected can result in 0 exit code and the separate invocation of the pip check may be needed. Also, the documentation for the pip check doesn't say much in what kind of scenarios the command may help. Even the new resolver section elaborates more on that. Maybe worth improving/extending? Thanks

[uranusjr]

Now that people use Docker extensively and relying on Docker layers caching heavily, installing dependencies in multiple steps is normal. But installing broken transitive dependencies silently is probably not normal.

I am curious about this. What scenario are using using Docker, and rebuild from a cached layer, and want to use pip to install packages into a conflicting environment? The idea of building with containers (to my understanding) is so the build can be deterministically reproduced from scratch, and installing multiple requirement files without resolving the conflicts between them seems to defeat the purpose. Could you elaborate more on the concrete use case here?

[pilosus]

and want to use pip to install packages into a conflicting environment?

I don't.

I just didn't know that after:

RUN pip install --no-cache-dir -r requirements-first.txt
...
RUN pip install --no-cache-dir -r requirements-second.txt

I need to pip check too. I expected pip install to do the consistency checks for me.

[pradyunsg]

Coolieo. I’m gonna consolidate this into #9094 now; I think the intended next steps are documented there, and you’ve got a working setup until then. :)