Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support git subfolder (#755) #2242

Conversation

finswimmer
Copy link
Member

This PR implements the possibility to add git dependencies, where the package is located within a subdirectory of the repository.

This was orginaly implemented in #1822, but needs rework due to the new poetry_core package.

Merging this PR requires merging of python-poetry/poetry-core#9 in poetry_core as well.

The path to the subdirectory can be specified as follows:

poetry add "git+https://github.com/demo/project_in_subdirectory.git?subdirectory=pyproject-demo"

Where pyproject-demo is a subfolder relative to the root after checkout the repository. This can be combined with the revision tag:

poetry add "git+https://github.com/demo/project_in_subdirectory.git#dev?subdirectory=pyproject-demo"

To specify the subdir in the pyproject.toml a new argument subdirectory was introduced:

mypackage = {git = "https://github.com/demo/project_in_subdirectory.git", rev = "dev", subdirectory = "pyproject-demo"}

To implement this feature is was necessary to checkout the repository first to a temporary folder and copy the desired folder to it's destination. For this a new DownloadTmpDir class was implemented, which avoids downloading from the same git source multiple times, during run time.

I would like to get some feed back from people, who can test this on a real world use case. Especially if this all works as expected.

Implements: #755

Pull Request Check List

This is just a reminder about the most common mistakes. Please make sure that you tick all appropriate boxes. But please read our contribution guide at least once, it will save you unnecessary review cycles!

  • Added tests for changed code.
  • Updated documentation for changed code.

…g properly

new (poetry-schema): add subdirectory property to git-dependency
new (pip_installer): git repositories are cloned to a tmp folder, to be able to copy only subdirectory to dest
new (locker): write to and read from `poetry.lock` the subdirectory value
new (provider): take the subdirectory value into account when looking for packing
new (package): new property `source_subdirectory`
new (console.commands.init): git subdirectory can be provided in url when using `poetry add`
… multiple times from the same source during run
…ng confusion to poetries repository caching system

change (utils.temp): renamed DownloadCache to DownloadTmpDir
fix (installed_repository): Catch CalledProcessError when trying to fetch revision
@finswimmer finswimmer force-pushed the issue-0755-git-subdirectory-for-core branch from 826d9dc to 8fec715 Compare April 4, 2020 06:04
fix (test_lock): comment out test that rely on changes in poetry_core
@pwhissell
Copy link

what is the status on this? Is there an E.T.A. ?

@finswimmer
Copy link
Member Author

Hello @pwhissell ,

we are still discussing how to test changes in poetry that rely on changes in poetry-core made at the same time. There are some ideas about it, but we don't have the time until now to establish such a pipeline. So unfortunately this PR is blocked until we solved the testing problem.

fin swimmer

@pwhissell
Copy link

Thank you for your quick resopnse @finswimmer
Are you guys prioritizing that testing pipeline for its implementation in 2020 or is it further down the priority queue?

@pwhissell
Copy link

In version 20.2.2, pip now supports this out of the box
https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support

@shaunc
Copy link

shaunc commented Sep 12, 2020

@finswimmer -- couldn't you push the changes to poetry core separately (with tests), and then push the rest afterwards? Don't let the bureaucracy get you down! :)

@finswimmer
Copy link
Member Author

finswimmer commented Sep 15, 2020

@shaunc Yes and no :) If I want to go that way, the pushed code to core must also be released. Once this is released I could push the code the poetry.

We have to find a better solution and I'm sure we will in the near future. We will run in this problem more often whenever a new feature is implemented in poetry.

@shaunc
Copy link

shaunc commented Sep 15, 2020

One solution over in javascript land is to use a monorepo, so that releases are automatically coordinated. That use case is supported, in part, by this very PR! :)

@kpeterson-sf
Copy link

Looking forward to this one. Solves a huge issue for me.

@danihodovic
Copy link

Useful feature for monorepos that contain a collection of packages such as https://github.com/man-group/pytest-plugins

@rubenlopezlozoya
Copy link

Guys, any updates on this?

@finswimmer
Copy link
Member Author

I'm closing this PR and the related in python-poetry/poetry-core#9 for the following reasons:

  • poetry's code base has changed a lot. I guess it would be easier to start from scratch then trying to adopt the current implementation
  • I'm not aware of any PEP that says how to describe a git dependency located in a subfolder

@finswimmer finswimmer closed this Aug 3, 2021
@tigerhawkvok
Copy link

@finswimmer If you specifically want to check for git subfolders, looking at submodules or subtrees is probably the right way to go

Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants