indicating package version of code within a tar

[tisdall]

This is not a code bug but a possible documentation issue. I'm trying to include a local tar file in my requirements.txt. All the documentation I've read simply mentions adding "#egg=[project_name]" but when I do only that the package will fail to install an updated version. I stumbled upon doing "#egg=[project_name]==[version]" and that seems to compare the currently installed version with version specified and install if they don't match. It also seems to properly understand if the local one matches that version and does nothing in that case too. However, I can't seem to find any documentation stating this is an intended function so I'm leery to use it.

To clarify with an example:

my_project 1.0 is already installed and I have a tar containing 1.1 . My requirements.txt file has my_project.tar.gz#egg=my_project. Pip will say that 'my_project' is already installed and skip over it.

If I change my requirements.txt to include my_project.tar.gz#egg=my_project==1.1, pip seems to then recognize that this version is different then the currently installed one and installs it. Also, subsequent pip install -r requirements.txt properly skip over install as the currently installed version matches the one in the requirements.txt .

[chrahunt]

Hello,

I checked and also couldn't find any docs that support this usage.

I'm hesitant to elevate this to the level of "supported", because of our intent to eventually deprecate the #egg= syntax in favor of PEP 508 (see #1289 for deprecation intent and PEP 440 for examples of direct references), which does not currently support specifying package version for direct URLs.

Alternatives that could work for your use case:

1. Name your project tar.gz as an sdist (my_project-1.1.tar.gz), then invoke pip with --find-links /path/to/parent/dir/of/sdist --upgrade and it should recognize and install newer versions from that directory. --find-links works in requirements files.
2. Invoke pip with --force-reinstall - this would be useful if you rebuild to the same version several times.

Also in case it saves some trouble, please note that relative paths in direct references are not implemented yet.

[tisdall]

@chrahunt I'm not sure I understand. Maybe it'd help to use my specific example. I have the following entry:
file:./local_src/python-service-clients-1.6.0.tar.gz#egg=service-clients[postgres,mongo]==1.6.0

It seems like PEP-508 would want it restructured into:
service-clients[postgres,mongo]==1.6.0 @ file:./local_src/python-service-clients-1.6.0.tar.gz

But, that results in "ERROR: Invalid requirement" and "Hint: It looks like a path." (maybe due to that relative path?)

Using what I have seems to work, but I'm just concerned about a newer version of pip breaking it. The fact that I can't find any documentation worries me.

I managed to do this:

>>> from pip._internal.req.constructors import parse_req_from_editable
>>> x = parse_req_from_editable('file:./local_src/python-service-clients-1.6.0.tar.gz#egg=service-clients[postgres,mongo]==1.6.0')
>>> x.requirement
<Requirement('service-clients[mongo,postgres]==1.6.0')>
>>> x.requirement.extras
{'mongo', 'postgres'}
>>> x.requirement.name
'service-clients'
>>> x.requirement.specifier
<SpecifierSet('==1.6.0')>
>>> x.link
<Link file:///./local_src/python-service-clients-1.6.0.tar.gz#egg=service-clients[postgres,mongo]==1.6.0>

Everything looks okay except for that link as that would resolve to an absolute path. However, it seems to currently work in 19.3.1 despite that.

[chrahunt]

Thanks for the examples, that helps!

It seems like PEP-508 would want it restructured into:
service-clients[postgres,mongo]==1.6.0 @ file:./local_src/python-service-clients-1.6.0.tar.gz

But, that results in "ERROR: Invalid requirement" and "Hint: It looks like a path."

This is expected, because versions are not allowed in direct references.

Using what I have seems to work, but I'm just concerned about a newer version of pip breaking it. The fact that I can't find any documentation worries me.

You are right to be concerned - if it is not documented then it is not technically supported and may change without notice (likely accidentally)!

---

To your original post

This is not a code bug but a possible documentation issue.

I'm hesitant to consider this a documentation issue, since updating the documentation to include this means that it is supported and we may not want to do that. The reason is that we are intending to deprecate this syntax in favor of PEP 440 direct references, which do not support specifying the package version.

I'm trying to include a local tar file in my requirements.txt.

All the documentation I've read simply mentions adding "#egg=[project_name]" but when I do only that the package will fail to install an updated version.

To satisfy your use case I do not think you need #egg=name or even #egg=name==version. Please try a requirements file like this:

--find-links ./local_src
python-service-clients==1.1

and run pip install -r requirements.txt. It should do you want:

1. upgrade to 1.1 if 1.0 is installed
2. do nothing when the same version is already installed

Please let me know if that is clear and works for you.

[tisdall]

I'm hesitant to consider this a documentation issue ...
I think the issue was that I didn't know where to look because I was used to adding #egg= to URLs and went down that road looking for the answer.

I wasn't sure I totally understood the --find-links but I see that it's looking at the filename to parse the package name and current version (following the PEP 508). I just have to rename 'python-service-clients-1.6.0.tar.gz' to 'service-clients-1.6.0.tar.gz' and it seems to work as expected. So, my specific problem is solved, but I'd still like to understand how to properly add external packages such as ones on git.

Thank you so much for the help!

[tisdall]

hmm.. I'm using pip-compile and unfortunately it translates the --find links ./local_src into an absolute path which simply won't work for our needs. I'll file an issue with pip-tools .

[tisdall]

--find-links file:./local_src works in pip-compile, however it still says "Looking in links: file:///./local_src" which worries me.

[uranusjr]

--find-links file:./local_src works in pip-compile, however it still says "Looking in links: file:///./local_src" which worries me.

There are a few quirks in urllib.parse (which is used to parse the file: URI), see discussion at pypa/packaging#120. Unfortunately this also means that you probably shouldn’t use relative paths with file:// in the long run.

[tisdall]

@uranusjr - my understanding is that the RFC defining file: URLs do not include relative paths. So, yes, it's the wrong thing to use, but unfortunately the only thing that currently works in pip-compile. Whenever jazzband/pip-tools#204 is resolved then I'll drop the file:.