Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv pip compile chokes on requirements files with explicit file:... requirements #3934

Closed
Julian opened this issue May 31, 2024 · 6 comments · Fixed by #3940
Closed

uv pip compile chokes on requirements files with explicit file:... requirements #3934

Julian opened this issue May 31, 2024 · 6 comments · Fixed by #3940
Assignees
Labels
bug Something isn't working compatibility Compatibility with a specification or another tool

Comments

@Julian
Copy link

Julian commented May 31, 2024

Given a trivial (i.e. empty) package foo, and a requirements.in containing:

file:.#egg=foo

(i.e. simply saying "install this package"), uv pip compile requirements.in crashes with:

⊙  uv cache clean && uv pip compile --verbose requirements.in                                                                                                                                                                                                                   julian@Airm
Clearing cache at: /Users/julian/Library/Caches/uv
Removed 578 files (19.3MiB)
DEBUG Starting interpreter discovery for any Python
DEBUG Looking for exact match for request any Python
DEBUG Searching for Python interpreter in all sources
DEBUG Found CPython 3.12.3 at `/Users/julian/.local/bin/python3` (search path)
DEBUG Using Python 3.12.3 interpreter at /opt/homebrew/opt/python@3.12/bin/python3.12 for builds
DEBUG Using registry request timeout of 30s
DEBUG Acquired lock for `/Users/julian/Library/Caches/uv/built-wheels-v3/path/2d0737fc0df579d6`
error: Failed to read from the distribution cache
  Caused by: failed to query metadata of file `/Users/julian/Desktop/foo/.#egg=foo`
  Caused by: No such file or directory (os error 2)

pip-compile of course still works:

⊙  pip-compile --strip-extras requirements.in                                                                                                                                                                                                                                   julian@Airm
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
#    pip-compile --strip-extras requirements.in
#
file:.#egg=foo
    # via -r requirements.in

though uv is unable to install what it produces, even though pip can:

⊙  uv venv venv                                                                                                                                                                                                                                                                 julian@Airm
Using Python 3.12.3 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: venv
Activate with: source venv/bin/activate

~/Desktop/foo is a git repository on main 
⊙  uv pip install --python venv/bin/python -r requirements.txt                                                                                                                                                                                                                  julian@Airm
error: Failed to read from the distribution cache
  Caused by: failed to query metadata of file `/Users/julian/Desktop/foo/.#egg=foo`
  Caused by: No such file or directory (os error 2)

~/Desktop/foo is a git repository on main 
⊙  python3.12 -m venv venv-venv                                                                                                                                                                                                                                                 julian@Airm
venv-ven%                                                                                                                                                                                                                                                                                   
~/Desktop/foo is a git repository on main 
⊙  venv-venv/bin/python -m pip install -r requirements.txt                                                                                                                                                                                                                      julian@Airm
Processing /Users/julian/Desktop/foo (from -r requirements.txt (line 7))
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: foo
  Building wheel for foo (pyproject.toml) ... done
  Created wheel for foo: filename=foo-0.1.dev1+g25e84d5-py3-none-any.whl size=2679 sha256=3185e32701885b76635d347608f2c457e39d2ce15c05099bc1c5100f47517696
  Stored in directory: /private/var/folders/8x/h479j0p15cq6l3m2r26kfdkm0000gn/T/pip-ephem-wheel-cache-o_zbhzsx/wheels/8b/19/c8/73a63a20645e0f1ed9aae9dd5d459f0f7ad2332bb27cba6c0f
/Users/julian/Desktop/foo/venv-venv/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:73: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
  return cls(files, info_location)
Successfully built foo
Installing collected packages: foo
Successfully installed foo-0.1.dev1+g25e84d5

This is on macOS with:

⊙  uv --version                                                                                                                                                                                                                                                                 julian@Airm
uv 0.2.5

where it seems some parsing is being done but the fragment hasn't been stripped.

(#3180 is related, as it's there again that this manifests for me, I once again cannot output something which both uv and pip are happy with, being on an older uv when first running the compile managed to get a compiled requirements file which as above uv itself could then not install)

Julian added a commit to bowtie-json-schema/bowtie that referenced this issue May 31, 2024
uv pip install with a file: URI has regressed on slightly newer uv.

See astral-sh/uv#3934.

This reverts commit 1d03249.
@charliermarsh
Copy link
Member

We can strip it, but why include the egg fragment at all?

@Julian
Copy link
Author

Julian commented May 31, 2024

That's the standard (to pip, it's not in any PEP AFAIK) way to indicate what the name of your package is. In which cases at this point in 2024 is that needed rather than falling back to parsing it out of pyproject.toml once building the wheel starts I don't know, but as far as I recall it's always been encouraged to include.

@Julian
Copy link
Author

Julian commented May 31, 2024

https://pip.pypa.io/en/stable/cli/pip_install/#working-out-the-name-and-version is the pip docs though they don't elaborate any further really.

@charliermarsh
Copy link
Member

I'll fix this case but note: https://pip.pypa.io/en/stable/topics/vcs-support/#url-fragments

pip also looks at the egg fragment specifying the “project name”. In practice the egg fragment is only required to help pip determine the VCS clone location in editable mode. In all other circumstances, the egg fragment is not necessary and its use is discouraged.

@charliermarsh charliermarsh self-assigned this May 31, 2024
@charliermarsh charliermarsh added bug Something isn't working compatibility Compatibility with a specification or another tool labels May 31, 2024
@Julian
Copy link
Author

Julian commented May 31, 2024

That section seems to only be referring to VCS URLs, though I'm sure you're right to point it out as it seems unlikely that it should be any more needed for archive or path URLs -- but given pypa/pip#1289 still being open and that more broadly as an end user it's been way easier over the years to find an incantation that works and never change rather than keep track of what's what, I'm quite glad that you're interested in the fix nonetheless.

@charliermarsh
Copy link
Member

My read of that thread and the Python Discuss thread is that it's only necessary at all for editable VCS URLs though I don't fully understand why it's necessary even in that case, and that the goal is to deprecate it entirely. I only mention this because I don't want others to read this issue and think that #egg is encouraged either here or in pip.

Regardless, fixed in #3940, since it's a legitimate bug on our end!

charliermarsh added a commit that referenced this issue May 31, 2024
charliermarsh added a commit that referenced this issue May 31, 2024
## Summary

This will help prevent bugs like #3934 by unifying the implementations
for editables and non-editable unnamed requirements. Specifically, both
of these now go through the same parsing paths and use the same struct
representations (with the exception that the editable flag is flipped in
the first case):

```
-e ./foo/bar
./foo/bar
```

We also now support PEP 508 in editable URLs. It turns out this is just
a limitation in pip, so it's correct to support it. For example, this
now works:

```
-e black[d] @ file://${PROJECT_ROOT}/scripts/packages/black_editable
```

Closes #3941.

Closes #3942.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working compatibility Compatibility with a specification or another tool
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants