Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Generate requirements.txt from uv.lock #6007

Closed
zeevro opened this issue Aug 11, 2024 · 18 comments · Fixed by #6778
Closed

Feature request: Generate requirements.txt from uv.lock #6007

zeevro opened this issue Aug 11, 2024 · 18 comments · Fixed by #6778
Assignees
Labels
enhancement New feature or improvement to existing functionality

Comments

@zeevro
Copy link

zeevro commented Aug 11, 2024

It would be beneficial to me (and I think in general) to be able to generate a requirements.txt file from the uv.lock file. It would be useful for installing requirements without having to resolve dependencies using a different installer in environments where uv is not installed, and also for having other tools that don't support the uv.lock file format (e.g. Dependabot) be able to know the project's pinned dependencies.

@charliermarsh charliermarsh added enhancement New feature or improvement to existing functionality preview Experimental behavior labels Aug 11, 2024
@charliermarsh
Copy link
Member

I think something like this is reasonable but it probably won't happen immediately. (PEP 751 is ultimately a better solution but I'd still support requirements.txt in the short-term.)

@zeevro
Copy link
Author

zeevro commented Aug 12, 2024

Also, we already have a way to generate requirements.txt in the form of uv pip compile but it's rather tricky to have that match my uv.lock, especially if it's done after a while and some dependencies have had new versions released.

@zanieb
Copy link
Member

zanieb commented Aug 12, 2024

It doesn't really seem like a great outcome for users to have both a uv.lock and requirements.txt though

@Samuel-Therrien-Beslogic

It doesn't really seem like a great outcome for users to have both a uv.lock and requirements.txt though

If using uv sync which creates a uv.lock file in a Docker, does uv sync take care of setting up references to the venv correctly ?
https://docs.astral.sh/uv/guides/integration/docker/#installing-a-package makes no mention of uv sync or uv.lock

@zanieb
Copy link
Member

zanieb commented Aug 14, 2024

Can you clarify what you mean by "references to the venv"?

I haven't written documentation for using the new APIs in Docker yet.

@Samuel-Therrien-Beslogic
Copy link

Samuel-Therrien-Beslogic commented Aug 14, 2024

Can you clarify what you mean by "references to the venv"?

I'm talking about this section of the doc:

To use the system Python environment by default, set the UV_SYSTEM_PYTHON variable:

ENV UV_SYSTEM_PYTHON=1

Alternatively, a virtual environment can be created and activated:

RUN uv venv /opt/venv
# Use the virtual environment automatically
ENV VIRTUAL_ENV=/opt/venv
# Place entry points in the environment at the front of the path
ENV PATH="/opt/venv/bin:$PATH"

I'm assuming if I use RUN uv sync --locked instead (to make sure the installation respects the lock file), that I don't need to mess around with venv setup ? (I just need to update my docker entry point command to use uv run instead of python, ref to our current transition to UV https://github.com/BesLogic/releaf-canopeum/pull/195/files#diff-1fc9c5328b5af6b1065537cb89aa7298eb7ba1d338b5bee8fd07553b6f56576f )

I haven't written documentation for using the new APIs in Docker yet.

That's a simple explanation as to why it's not referenced ^^


I'm just ensuring that the uv.lock file should be usable in Docker. Which removes the need for this feature request for me.

@zanieb
Copy link
Member

zanieb commented Aug 14, 2024

uv sync always creates a virtual environment right now, so yeah you'd either need to opt for configuring it (as described above) or use uv run instead of invoking python.

We may add support for syncing to a system environment (as requested in #5964) to make this a little simpler in CI and containers.

I'm just ensuring that the uv.lock file should be usable in Docker

Yeah it definitely is. I'll try to update the documentation soon.

@zanieb zanieb removed the preview Experimental behavior label Aug 20, 2024
@matmair
Copy link

matmair commented Aug 20, 2024

It doesn't really seem like a great outcome for users to have both a uv.lock and requirements.txt though

I have this use case right now; maybe the insight is useful. There are strict requirements only to deploy stable software in the deployment targets. That rules out uv on the deployment targets by our current rules. The devs still want to use uv and uv project management and it is pretty confusing to have the requirements.txt that is used for deployments and the uv.lock within the repo.

This can be solved by CI/CD but reducing steps and complexity is the signature of uv/ruff for me.
Loving the tools btw, y'all are doing gods work

@zanieb
Copy link
Member

zanieb commented Aug 20, 2024

What makes uv unstable?

Thanks for sharing your use-case! Glad you like the tools :)

@matmair
Copy link

matmair commented Aug 20, 2024

SemVer. I do not make the rules sadly.

@jfgordon2
Copy link

jfgordon2 commented Aug 22, 2024

I also am finding a need to produce a requirements.txt, but just for non-dev packages. The current path I have in our pipeline looks something like:

uv sync --no-dev
uv pip freeze > requirements.txt

Then we can check that requirements file against safety (or dependabot) for vulnerabilities, then do a uv pip install -r requirements.txt --target dist so we can package it up along with src into a zip file to deploy to a lambda function.

It definitely would be easier to operate directly against the uv.lock, but in the meantime converting that directly into a requirements.txt for compatibility with other tools would be great.

@chrisrodrigue
Copy link

chrisrodrigue commented Aug 22, 2024

+1 to what @jfgordon2 said.

I think supporting requirements.txt input and output is wise to maintain compatibility with legacy tooling that isn’t yet caught up with PEP 621.

I have documented one such use case for requirements.txt here: #6422

@phi-friday
Copy link

As you mentioned in #6429 , will there be a feature to create a requirements.txt including tool.uv.dev-dependencies?

@charliermarsh
Copy link
Member

Yeah, that would probably be included in the scope of this.

@zeevro
Copy link
Author

zeevro commented Aug 26, 2024

I'm using this as a make-shift solution for now.

uvx --from=yq tomlq -r '.package[]|.name+((.source|" @ "+(.url//"git+\(.git//empty)"))//"==\(.version)")' uv.lock >requirements.txt

@charliermarsh
Copy link
Member

Respect!

@SamEdwardes
Copy link

I like the idea of uv export!

I am curious what are the implications of doing this:

uv pip compile pyproject.toml -o requirements.txt

In this case is my uv.lock ignored and there is a risk the generated requirements.txt will be different?

@charliermarsh
Copy link
Member

In this case is my uv.lock ignored and there is a risk the generated requirements.txt will be different?

Yeah, the uv pip interface never touches the lockfile. You can use the uv pip workflow if you want, but you lose the benefits of the unified uv sync, uv lock, uv run APIs.

charliermarsh added a commit that referenced this issue Aug 29, 2024
## Summary

The interface here is intentionally a bit more limited than `uv pip
compile`, because we don't want `requirements.txt` to be a system of
record -- it's just an export format. So, we don't write annotation
comments (i.e., which dependency is requested from which), we don't
allow writing extras, etc. It's just a flat list of requirements, with
their markers and hashes.

Closes #6007.

Closes #6668.

Closes #6670.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement to existing functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants