-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pip 20.3+ and its new dependency resolver #1109
Comments
Pip 20.3 has just been released: As explained above, we won't be upgrading to it straight away, so apps using this buildpack will be unaffected for now. I'd imagine we'll try updating some time at the start of next year. |
pip 21 has now been released, which drops Python <3.5 support. To install the latest pip supported by the current Python there's this dummy package: https://github.com/graingert/pip-with-requires-python . |
@adamchainz Hi! That package is only needed for scenarios where someone is both using an environment with a very old pip version that does not support the new "requires Python version X" metadata (for example an older Linux distro's system pip) and also wants to In terms of updating the Python buildpack's pip to a newer version, it's on the list for later this year, however:
That said, I'm looking forward to being on newer Pip as soon as viable :-) |
yes, but also it has the serendipitous effect of installing "the latest pip supported by the current Python" |
This does not help us, since for performance reasons the Python buildpack intentionally installs the chosen pip version from the outset, rather than installing a bootstrap version and then upgrading it. |
@edmorley I'll present a case where the old resolver fails (and this affects multiple Heroku-hosted projects of mine). In the requirements structure:
the list of installed packages is just:
In other words, in the old resolver, if a transitive requirement ( The workaround is just to repeat any extras on the top-level requirement, despite the fact that they may not actually be required there. For what it's worth, I am requesting the option to enable newer versions of pip (with the new resolver) on an opt-in basis, though I certainly appreciate the reasons why you're hesitant to make this the default. |
@brianhelba Thank you for the example :-) Would an update to pip The only issue I can see is that |
I'd be happy to try it out.
I only started using the new resolver once it was released in |
Hmm I'm leaning towards updating to |
Opened #1192 for this. |
Updates pip from 20.1.1 to 20.2.4 for Python 2.7 and Python 3.5+. Python 3.4 continues to use 19.1.1, the last pip version to support it. We're not updating to pip 20.3+ yet, since those versions enable the new pip dependency resolver by default, which has compatibility implications, so needs additional UX work first as well as many upstream fixes as possible (see #1109). Changes: https://pip.pypa.io/en/latest/news/#v20-2-4 pypa/pip@20.1.1...20.2.4 Closes GUS-W-7944197.
Updates pip from 20.1.1 to 20.2.4 for Python 2.7 and Python 3.5+. Python 3.4 continues to use 19.1.1, the last pip version to support it. We're not updating to pip 20.3+ yet, since those versions enable the new pip dependency resolver by default, which has compatibility implications, so needs additional UX work first as well as many upstream fixes as possible (see #1109). Changes: https://pip.pypa.io/en/latest/news/#v20-2-4 pypa/pip@20.1.1...20.2.4 Closes GUS-W-7944197.
Hi, I’m a maintainer of pip and worked on the dependency resolver. I see the step still installs 20.2.4 by default, is there something we can do to help upgrading? |
@uranusjr Hi! Thank you for reaching out (and all your hard work on pip!). I'm very keen to update to a post 20.3 release once the rate of new resolver bugs/issues has settled. I've been checking the pip repo's issues and PR activity a few times a week, and it seems that has mostly occurred by now. There are a few new resolver fixes in 21.1, so I was thinking of updating to 21.1 once it's released. That said, pypa/pip/issues/9187 is still concerning as it means some users will have their builds hit the Heroku build timeout. Unless I add a wrapper around the pip invocation (that ends the pip install before the overall build timeout; which is problematic since the overall timeout is dynamic), I'll have no way to identify these cases and output messaging to users as to how to resolve. Which means I either don't output anything (and leave users to google or check the Heroku knowledge base articles after failures), or have to output a "using new pip resolver, visit I'll also need to catalogue the various non-timeout pip new resolver failure modes (specifically the error message strings that can occur), so the Python buildpack can check for them during the "analyse pip errors" step, and output useful suggestions. For example linking to the pip docs on resolving dependency conflicts, and also saying that users can temporarily disable the new resolver by adding |
Thanks for the feedback. The undetermined resolution time is definitely a big issue, and is really an issue for every dependency resolver. It is however very difficult to set a reasonable cap to resolution resources since it both is highly subjective and varies across scenarios—the same problems you have with a timeout applies to everything else as well. On the other hand though, if the new resolver takes a lot of time, the legacy resolver would just populate an environment that’s going to fail at runtime in most (if not all) scenarios. So using the legacy resolver is more or less punting the issue back to the user (which is of course a perfectly fine solution for a service). Pip intends to shift this complexity back to the user in the long run as well, since the user is really the only person that knows how long is too long for a resolution to take. This would bring Python’s packaging experience closer to ecosystems like e.g. Rust, where all versions are resolved ahead of time in a “lock file”, and the service would simply install packages listed in it. So instead of running But that’s probably not a useful thing for Heroku until the Python packaging ecosystem can produce a smooth-ist transition to the new workflow (and the pip version is no longer relevant after that; |
I've seen at least one package on PyPI that has recently switched to only shipping PEP660 style wheels (eg I guess time to now update? 😆 I'll also need to find a set of example failing cases with the new resolver, so I can add handling for them in this buildpack (eg looking for specific failure modes and suggesting that the user try |
For a dependency error (that would previously succeed in the legacy resolver), the new resolver would emit a message containing the string List of issues on the new resolver: https://github.com/pypa/pip/issues?q=is%3Aopen+is%3Aissue+label%3A%22C%3A+dependency+resolution%22 Currently there are mainly three categories the new resolver does not do well (not including
The latter two are both |
I'm still actively following pip releases and known issues for the new resolver. I was very much hoping that we could update to the 21.2.x releases, however there appear to be further regressions, for example: I would love to see pypa/pip#10415 fixed in a release soon (there is a WIP PR), given it sounds like it would help with that regression. |
Please note that the author of both pypa/pip#10415 and the associated draft PR explicitly (and very correctly) states the fix is not likely to help many cases affected in pypa/pip#10201. |
Ah I was thinking of pypa/pip#10201 (comment) which mentions it includes a fix for that (in addition to the new ordering approach) - sounds like the impact was minimal compared to the new ordering approach itself. Anyway, I mainly wanted to update this issue to say I hadn't forgotten about it, but am waiting for things like pypa/pip#10201 to be fixed. |
Just saw this issue, is there a particular set of problematic requirements or is this a general worry? I've put together a summary of what I think will be a big optimization for most real world dependency trees: pypa/pip#10479 You can test it with this tag if you have specific examples: Though I would appreciate if you could share those specific examples so I can test myself. |
@notatallshaw Hi! I don't have specific examples yet (I won't until we upgrade and people file support tickets), it's more that:
Thank you for working on pypa/pip#10479 -- it looks like a promising improvement that may fix pypa/pip#10201 and friends enough that we can finally update here :-) |
It's interesting though because Pip, 20.2 very much is able to create broken environments that may not work at all. So "just works" is a tricky one, the longer I've been looking at this issue the more I'm surprised pip ever worked in the first place with any moderately complex dependencies.
I also agree, which is why I've been thinking about options like this: pypa/pip#10417 |
I agree the old resolver can lead to broken environments. The issue is that for those Heroku app environments that were actually non-functioning, users will have self-corrected their requirements so that their application works months/years ago. Those that remain are those whose |
Pip 21.3 was released yesterday: Of note...
This will help with a significant proportion of the cases that would have exceeded the build timeout.
This will help slightly with the UX issue - that is, users of this buildpack not knowing why package installation is seemingly taking a long time or timing out. We may still need some additional messaging in the buildpack build log output, but this means we won't have to explain quite as much - and does a better job than we could, since it's only shown when needed (whereas for our messaging we would have to show it unconditionally given we can't just show it at the end due to dynamic build timeouts). |
Pip 21.3 regressions that may mean we need to wait for 21.3.1: |
Update pip from 20.2.4 to: - 20.3.4 for Python 2.7 and 3.5 - 21.3.1 for Python 3.6+ Of note Pip 20.3+ includes the new dependency resolver (only enabled by default when using Python 3+). This new dependency resolver is more strict, see: https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020 https://pip.pypa.io/en/stable/topics/dependency-resolution/ Release notes: https://pip.pypa.io/en/stable/news/#v21-3-1 Changelog: pypa/pip@20.2.4...21.3.1 The new versions of pip have been synced to S3 using: ``` $ pip download --no-cache pip==20.3.4 ... Saved ./pip-20.3.4-py2.py3-none-any.whl Successfully downloaded pip $ pip download --no-cache pip==21.3.1 Collecting pip==21.3.1 ... Saved ./pip-21.3.1-py3-none-any.whl Successfully downloaded pip $ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "*.whl" --dryrun (dryrun) upload: ./pip-20.3.4-py2.py3-none-any.whl to s3://heroku-buildpack-python/common/pip-20.3.4-py2.py3-none-any.whl (dryrun) upload: ./pip-21.3.1-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-21.3.1-py3-none-any.whl $ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "*.whl" upload: ./pip-20.3.4-py2.py3-none-any.whl to s3://heroku-buildpack-python/common/pip-20.3.4-py2.py3-none-any.whl upload: ./pip-21.3.1-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-21.3.1-py3-none-any.whl ``` Closes #1109. GUS-W-8493316.
Update pip from 20.2.4 to: - 20.3.4 for Python 2.7 and 3.5 - 21.3.1 for Python 3.6+ Of note Pip 20.3+ includes the new dependency resolver (only enabled by default when using Python 3+). This new dependency resolver is more strict, see: https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020 https://pip.pypa.io/en/stable/topics/dependency-resolution/ Release notes: https://pip.pypa.io/en/stable/news/#v21-3-1 Changelog: pypa/pip@20.2.4...21.3.1 The new versions of pip have been synced to S3 using: ``` $ pip download --no-cache pip==20.3.4 ... Saved ./pip-20.3.4-py2.py3-none-any.whl Successfully downloaded pip $ pip download --no-cache pip==21.3.1 Collecting pip==21.3.1 ... Saved ./pip-21.3.1-py3-none-any.whl Successfully downloaded pip $ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "*.whl" --dryrun (dryrun) upload: ./pip-20.3.4-py2.py3-none-any.whl to s3://heroku-buildpack-python/common/pip-20.3.4-py2.py3-none-any.whl (dryrun) upload: ./pip-21.3.1-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-21.3.1-py3-none-any.whl $ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "*.whl" upload: ./pip-20.3.4-py2.py3-none-any.whl to s3://heroku-buildpack-python/common/pip-20.3.4-py2.py3-none-any.whl upload: ./pip-21.3.1-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-21.3.1-py3-none-any.whl ``` Closes #1109. GUS-W-8493316.
So I investigated this approach as part of #1259, however unfortunately the set of pip options that can be configured via As such, the only way to provide a user-controllable toggle would be to do so via custom buildpack environment variable or some non-standard In addition, it's now been a year since upstream pip has been using the new resolver, meaning that most of the commonly encountered dependency issues in the wild have been fixed. Given all of that, for now I've decided not to add an opt-out, especially since apps/dependencies will eventually need to adapt regardless (pip will be removing support for the new resolver in a future release), and that those that need a workaround in the meantime can pin to the previous buildpack release ( In the changelog entry for this change, I've linked to the pip docs on resolving |
It's possible to add |
@uranusjr Yeah I'd contemplated filing an issue/opening a PR, however it would presumably now have to wait until the v22 pip release in 3-4 months, by which time the need for this will have reduced on our side. |
The soon to be released pip 20.3 comes with a new dependency resolver that's enabled by default:
https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020
This dependency resolver is a great improvement for pip, however it is by design more strict so in some cases will cause pip install failures in apps whose builds previously succeeded. Some of these failures will be fixed by upstream package maintainers fixing their package's declared dependencies - others may require changes by end users of apps using this buildpack.
As such the Python buildpack will not be enabling the new resolver straight away (in order to give the community time to fix upstream broken packages and/or discover any last bugs in the resolver itself), and when it is enabled, we'll have documentation for how to switch back to the legacy resolver (using
requirements.txt
option flags), a changelog entry and likely error message handling in the buildpack itself directing users to the docs if a resolver related error is encountered.Note: The Python buildpack pins the version of pip it uses, so users of this buildpack won't see any change in behaviour on pip 20.3 release day, only when we explicitly upgrade.
The text was updated successfully, but these errors were encountered: