Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev-dependencies in workspace pyproject.toml not installed with uv run #7487

Open
KilianMichiels opened this issue Sep 18, 2024 · 8 comments
Labels
question Asking for clarification or support

Comments

@KilianMichiels
Copy link

We have a mono repository with a couple of packages and services. I wanted to unify the dev-dependencies for all packages (each of them have their own pyproject.toml file) since they are largely the same dependencies.

I noticed when moving the dev-dependencies from the package pyproject.toml files to the workspace one in the root folder, that these dependencies are no longer installed when I run uv run in a package folder (e.g., to run a test script with pytest).

Is this behaviour expected? I assumed dependency resolution would take into account all dev-dependencies in all pyproject.toml files inside the workspace.

@charliermarsh
Copy link
Member

To confirm my understanding: you want to define the dev-dependencies in a root pyproject.toml, and then have them available when running commands like uv run --package some-member -- is that right?

@charliermarsh charliermarsh added the question Asking for clarification or support label Sep 18, 2024
@KilianMichiels
Copy link
Author

KilianMichiels commented Sep 18, 2024

Hi @charliermarsh, that is correct! Since I'm working in a workspace where things like pytest, coverage, ruff, etc. are all shared, it seemed like the logical thing to do.

I added an example below to explain my use case a bit further:

Directory structure:

<root>
  -- pyproject.toml
  -- scripts
  -- other_stuff
  -- my-package
      -- pyproject.toml
      -- my-package
      -- tests

In my root directory, I have this pyproject.toml:

[project]
name = "my-project"
version = "0.1.0"
requires-python = ">=3.10,<3.13"
description = "My simple project"

[tool.uv]
# NOTE: These dependencies are not picked up when running
#       in a package folder. Therefore I kept them in the
#       pyproject.toml of each package for now.
dev-dependencies = [
    "bandit>=1.7.9",
    "pytest-cov>=5.0.0",
    "pytest-env>=1.1.3",
    "pytest-xdist>=3.6.1",
    "pytest>=8.3.2",
    "ruff>=0.6.5",
]

[tool.uv.sources]
my-package = { workspace = true }

[tool.uv.workspace]
members = [
    "my-package",
]

Then in my package folder I have this pyproject.toml:

[project]
name = "my-package"
version = "0.1.0"
requires-python = ">=3.10,<3.13"
description = "My simple package"

[tool.uv]
# I would expect these to not be needed anymore as they are defined in the dev-dependencies of the workspace.
dev-dependencies = [
    "bandit>=1.7.9",
    "pytest-cov>=5.0.0",
    "pytest-env>=1.1.3",
    "pytest-xdist>=3.6.1",
    "pytest>=8.3.2",
    "ruff>=0.6.5",
]

Then I'd like to be able to do for example:

cd my-package
uv run ruff check my-package/

And this gives me the error (on a cold run, so no uv sync was run yet, inside a Docker Gitlab CI) if I comment out the dev-dependencies in the package pyproject.toml and only have the ones in the root file:

> error: Failed to spawn: `ruff`
  >   Caused by: No such file or directory (os error 2)

Additionally, it might be nice to be able to add common dependencies to the root workspace pyproject.toml and add more specific ones to the package pyproject.toml.

Edit: Added my-package = { workspace = true } and members to the root pyproject.toml to better represent my current implementation.

@KilianMichiels KilianMichiels changed the title dev-dependencies in workspace pyproject.toml not installed with uv sync dev-dependencies in workspace pyproject.toml not installed with uv run Sep 18, 2024
@BurntSushi
Copy link
Member

I think I like the status quo, where you can do dep = { workspace = true } to inherit the specification for dep, so you don't need to repeat the version constraints, but still requires opt-in to add the dev-dependency in the first place. I wouldn't expect all packages in all workspaces to need the same dev-dependencies in all cases, so if we made this case work as expected, I'd also expect some way to opt out of it. Another approach here might be to add an option for dev-dependencies as to whether they should be "forcefully" inherited in packages contained in the workspace.

@KilianMichiels
Copy link
Author

I think I like the status quo, where you can do dep = { workspace = true } to inherit the specification for dep, so you don't need to repeat the version constraints, but still requires opt-in to add the dev-dependency in the first place.

Is this the functionality under [tool.uv.sources] you are talking about? As I thought this solution was mainly to set your own packages as part of the workspace environment? I tried adding ruff = {workspace = true} in my root pyproject.toml but got a Package is not included as workspace package in 'tool.uv.workspace' error...which makes sense.

I wouldn't expect all packages in all workspaces to need the same dev-dependencies in all cases, so if we made this case work as expected, I'd also expect some way to opt out of it.

I agree this is not by default the case. I believe the proposed solution should indeed be an opt-in if you add specific dev-dependencies to your root pyproject.toml. Then only those would be shared across the workspace. Not doing so, essentially results in the current working implementation where only dev-dependencies in the package's pyproject.toml are picked up. But since I'm quite new to uv I might be missing something here. :)

Another approach here might be to add an option for dev-dependencies as to whether they should be "forcefully" inherited in packages contained in the workspace.

I don't think I have enough experience with uv to provide any useful input here, but here's my 2 cents :) On the one hand this would make the settings more explicit which is nice. On the other hand, it might add more complexity to the pyproject.toml files and one might lose the overview of which packages force this behaviour if it should be set in the package's pyproject.toml. If it's a single setting in the root pyproject.toml I think it could be nice, but I don't see the added benefit of having both the setting and common dev-dependencies (adding them to the root implies having this setting on true as far as I can see).

@BurntSushi
Copy link
Member

The issue is when you add a dev-dependency to the root but don't want it to be included in a sub-package. That's how dev-dependencies and normal dependencies work today. Correct me if I'm wrong, but I believe you are suggesting that dev-dependencies should work differently: that a dev-dependency in the workspace root pyproject.toml should be automatically inherited by all workspace packages.

The { workspace = true } thing applies to your my-package example, not the workspace root. It's saying, "use the value defined in the workspace root pyproject.toml."

@KilianMichiels
Copy link
Author

Oh now I understand! In that case, my suggestion indeed works differently than the current behavior. So this might be quite a big change to add and I don't think I have a proper overview of the impact of such a change..

I'm only looking for a way to only define some dev-dependencies once, if they are shared across all packages / services / ... in the workspace. Like ruff! :) But as far as I can see, there is no easy way to do this? The only solution I could find was adding ruff to all my dev-dependencies for all pyproject.toml files. If there is a better way, that would be more than welcome! :)

@BurntSushi
Copy link
Member

So one thing it looks like I got wrong is that { workspace = true } is not working as I had expected it to. From the docs:

The workspace = true key-value pair in the tool.uv.sources table indicates the bird-feeder dependency should be provided by the workspace, rather than fetched from PyPI or another registry.

That's saying the dependency should be in the workspace. I had gotten this wrong and thought it meant that it "inherits" what dependency specification is in the workspace root. So I think the situation is a little worse than I was painting: you do have to repeat not just the dependency but the version constraint.

I don't have a strong opinion about whether there's an option to make dev-dependencies apply everywhere, but I do kind of think it should be opt-in. Or maybe even opt-in on a dependency-by-dependency basis in the workspace root.

@KilianMichiels
Copy link
Author

I really like the idea of having the opt-in on a dependency-by-dependency basis in the workspace root! However, looking at the docs I don't see a ready-to-use solution that aligns neatly with PEP 508 and allows you to set a flag or something for each dev-dependency? So perhaps a global opt-in under [tool.uv] or [tool.uv.dev-dependencies] would already be an easy win until there is a need for a more fine-grained solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Asking for clarification or support
Projects
None yet
Development

No branches or pull requests

3 participants