Report language, abi, platform tags when "DistributionNotFound: No matching distribution" occurs

[romanows]

• Pip version: 9.0.1
• Python version: 2.7.7
• Operating System: Linux / CentOS 5

Description:

Recently, Tensorflow upgraded their wheels to require an ABI tag of "cp27mu" from the previous required ABI tag of "none". I didn't realize this and was puzzled when I could not install the wheel as I had previously done. I was getting a "DistributionNotFound: No matching distribution found for tensorflow" message. Even after looking at the wheel names on pypi, it looked like there was a wheel that matched my Python install.

The problem turned out to be that my Python binary has an associated ABI tag of "cp27m" which is not compatible with "cp27mu". I only realized this after putting some prints in the pip source.

It would be helpful for this information to be exposed by reporting the sets of (language, abi, platform) tags when their mismatch is the cause of a "no matching distribution" error message. I would say this should be printed by default, but I could see the argument for printing it only when the -v verbose command line option is specified. These details should not be shown when errors are due to a lack of distribution for a specified distribution name or when errors are due to a version mismatch.

From poking around, it looks like the tags for the current install are available from pep425tags.supported_tags, although it looks like there may be ways to specify other supported tags. These are checked in wheel.py/Wheel.supported(), although there may be other checks.

Perhaps the output could look like:

DistributionNotFound: No language, abi, and platform tag match found for tensorflow distribution.
Local pip install supports the following tags:
* language: 'cp27', 'py2', 'py22', 'py23', 'py24', 'py25', 'py26', 'py27'
* abi: 'cp27m', 'none'
* platform: 'any', 'linux_x86_64', 'manylinux1_x86_64'

What I've run:

$ pip install tensorflow

Collecting tensorflow
  Could not find a version that satisfies the requirement tensorflow (from versions: )
No matching distribution found for tensorflow

[pradyunsg]

Thanks for filing this issue!

Printing all the tags would not be very useful in every situation. That said, the output could definitely use some improvement.

[dansgithubuser]

I'm having similar troubles. Is there another way to ask pip what tags it is looking for?

[gar1t]

@pradyunsg It's indispensable information when you run into the problem that requires it. The fact that there are cases where it's not useful doesn't change that fact.

This information could be included in the --verbose output without impacting the default (completely uninformative) behavior. As it stands, it's practically impossible to troubleshoot these issues, which stops users dead in their tracks.

I'm happy to submit a PR for this if there's interest. But as this is June 2019 and this the issue was opened in early 2017, I'm wondering if there's some other blocker to implementing this reasonable and useful feature.

Or maybe I'm missing an option that provides this, in which case let's update and close this issue.

[pfmoore]

I'm wondering if there's some other blocker to implementing this reasonable and useful feature.

One thing that needs consideration is that the list of supported tags can be very long (I think I've heard comments suggesting that in some cases it can be hundreds of entries). How to report that sensibly is likely to be difficult ("verbose" doesn't mean pages of output which are rarely useful, there's still a need to be reasonable even under -vvv :-))

Note that the OP's proposed output isn't actually correct, in that you can't necessarily combine the 3 parts arbitrarily.

But if someone were to produce a PR, I don't think there's any reason in principle why it wouldn't be accepted.

[gar1t]

Please see #6575

I'll update NEWS and verify tests but I want to socialize this change to get feedback.

While there can be many tags, there can also be many other things in verbose logging. The many lines showing Skipping link... is great information on what is skipped, but without the tags list, it's impossible to know why the link was skipped. This is all critical information for understanding what candidates are being considered and, importantly, why a candidate is skipped.

[gar1t]

As there's no consensus at this point on how to solve this problem, I've created an external script that folks can use to debug candidate selection problems with a package/requirement.

https://github.com/gar1t/check-pip-install

Ideally this information would be provided by pip itself. Not being able to install a package is frustrating enough. Not having a reason why is that much more frustrating. As this issue is 2 and half years old, I can only assume that it's not a problem for the pip project. I can say from experience, it's a problem for users, but apparently not enough to drive work on this issue.

As hopefully a short term workaround, the check-pip-install script should help provide the info needed.

[pradyunsg]

As this issue is 2 and half years old, I can only assume that it's not a problem for the pip project.

It is. Just because it isn't solved immediately doesn't mean we don't care.

Do note that the maintainers volunteer their time so it's not really feasible to solve every problem, even ones with great rewards for the effort, in a reasonable amount of time. eg. see 988, 4575 and more.

[pradyunsg]

I'm wondering if there's some other blocker to implementing this reasonable and useful feature.

The blocker is someone willing to do the work here -- consensus building, design and implementation.

[pfmoore]

The blocker is someone willing to do the work here -- consensus building, design and implementation.

Just to support what @pradyunsg said here. The hard work isn't in writing the code, it's in getting agreement on the best solution, designing something that provides focused help for people that need it without inconveniencing people who don't, etc. As @gar1t has discovered, that isn't easy and takes time - time which is in very short supply for the pip developers (ironically, most of our available time is taken up with offering reviews, design help and support to people offering PRs). I can understand it's frustrating for someone to offer a patch and have it get bogged down in "big picture" design questions, but it's just as frustrating for the pip devs (who do care about fixing issues like this, in spite of what people claim) when PRs appear but then get stalled or abandoned because a simple first draft solution needs refinement before it's suitable.

[cjerdonek]

A couple comments:

1. Recently in this comment, I suggested the idea of adding a pip tags command for doing things like printing out the list of tags. As others have said, I think this list is too long to be included in a normal log message, even in verbose mode (which is why I suggested a separate command for this purpose).

2. Also, this PR that was recently merged might be of interest: Improve the debug log message when installing an incompatible wheel #6540 . It helps with the other side of the coin by displaying which tags each wheel would have matched with (which is a much shorter list).

[cjerdonek]

Also, in another comment I included some python that can be used to print out the list of tags: #6547 (comment)

Obviously it's not a permanent solution, but it can be used for debugging purposes in a pinch.

[xavfernandez]

I could not find an issue where it was mentioned before but I like the idea of a pip debug command (somewhat like your pip tags but more general) to diagnose pip issues:

• print pip version and python version/interpreter
• print supported pep 425 tags
• print installation path
• print read configuration
• inspect PATH issues
• inspect installed packages & metadata
• etc

(Possibly under different sub-commands)

[pradyunsg]

I suggested in #6575 that we should print the number of supported tags and a hint to getting the exact list of tags.

I think the most difficult question here is figuring out the CLI to be used.

[pradyunsg]

Hahah. I found the pip debug comment. Any guesses what one of the motivating examples for it was? ;)

#5813 (comment)

FTR - I think we should move to a new issue to discuss pip debug, if that's the way we want to expose this at the CLI level.

[gar1t]

pip developers, I've bumped this thread with a pull request that solves the problem. You're welcome.

Someone objected without proposing an alternative. Fine.

As a long time open source developer and contributor, I have no illusions about how this process works. You have my contribution - it's just one idea. As I'm not skilled at solving problems that don't exist (hypothetical complaints about having too many tags) I'm afraid my contributions to this thread are stalled.

As I've shown with an external project that can be used to solve this problem - we don't need to have a consensus to move the ball forward.

[pradyunsg]

The problem is not hypothetical -- I've demonstrated the same on the PR. Furthermore, alternatives have been discussed and mentioned. And a concern being raised without suggesting how to resolve that concern is fairly normal.

Anyway, I don't have the mental energy to explain why concensus building and addressing concerns raised by others is an important part of OSS (and many social systems) so I'll skip that.

Regardless, thanks for your contribution!

[cjerdonek]

I think a pip debug command would also work to list the current tags, though I do think we should reserve the ability to diagnose compatibility with an individual tag using the command (e.g. an individual wheel file name). I think the only question for the purposes of this issue is whether the list of current tags should be displayed by default for pip debug, or if it should require an additional flag or be behind a pip debug subcommand to display (because the output in this case is long).

[cjerdonek]

FYI, I posted a PR #6638 implementing an initial pip debug command. As was suggested above, it displays the list of compatible pep 425 tags. It also accepts the same --python-version, --platform, etc. options that pip install and download support so that one has a way to experiment and see what tags result in those cases, too.

[vorburger]

#6526, #7565 and #8831 (and the broader #10421) look like related newer issues to this older issue which perhaps could be duplicated to this.

[uranusjr]

I’m going to just close this in favour of #10421. This specific error message comes from the legacy resolver and is not going to be improved upon. The newer issue provides more relevant context toward the issue in its current state, against the current resolver implementation.