[Feature] Pip QOL commands -- EXPERIMENTAL #657
Conversation
|
Hi @lambdaclan, I'm a little hesitant about where this is headed. Rez isn't supposed to be a pip wrapper - there are other projects aiming to do that, such as https://github.com/myint/yolk. Specifically, it appears you can get a listing of PyPI versions of a package using Are we in danger of replicating the efforts of other packages, and do you think this is within the scope of rez? Bear in mind that rez was never intended to be a package manager specific to python (in fact it largely managed C++ packages when it was first written). Adding integration with pip did make sense however, since python packages are so widely used in VFX, and the large majority are already made available via pip. |
|
Hello Allan, yes I can see what you are saying we definitely do not want to be adding things to rez that are out of scope. The reason I labelled this PR experimental is because I am trying to see if such features would be beneficial in the greater scheme of things since most revolve around version checking, validations etc that can be useful when authoring rez packages with strict requirements. yolk does look like a nice alternative but basically I wanted to avoid external dependencies. I personally do not have any issues with not including such functionality intro rez-pip if it does not make sense. I will consult with @instinct-vfx once more and see if we have any particular needs for these changes and will get back to you. We can always extract it as an external standalone tool or as you mention above use a different package so there are many options. I will keep the PR open for now. What is important for us is #628 and #659 so I will focus on those for now. |
|
No worries. To be clear, I understand the need for the functionality that
some of these cli options represent - for example, determining the python
version(s) compatible with a given package. But I think it would be better
to concentrate on that functionality first and foremost, then consider
exposing it in the cli tool if that proves useful. The immediate aim of
rez-pip is to successfully convert a pip package and its requirements into
rez packages, with correct analogous rez dependencies.
Thx
A
…On Thu, Jul 11, 2019 at 11:15 AM Ira ***@***.***> wrote:
Hello Allan, yes I can see what you are saying we definitely do not want
to be adding things to rez that are out of scope. The reason I labelled
this PR experimental is because I am trying to see if such features would
be beneficial in the greater scheme of things since most revolve around
version checking, validations etc that can be useful when authoring rez
packages with strict requirements.
yolk does look like a nice alternative but basically I wanted to avoid
external dependencies.
I personally do not have any issues with not including such functionality
intro rez-pip if it does not make sense. I will consult with @instinct-vfx
<https://github.com/instinct-vfx> once more and see if we have any
particular needs for these changes and will get back to you. We can always
extract it as an external standalone tool or as you mention above use a
different package so there are many options.
I will keep the PR open for now. What is important for us is #628
<#628> and #659
<#659> so I will focus on those for
now.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#657>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMOUSQRGRI7EXZ6572LY43P62CSPANCNFSM4H6XG6TA>
.
|
|
Sorry for the silence here. I am fine with splitting this off and focus on functionality required for rez-pip itself only first. rez.pip does have a high priority for us so i am all for getting this on track asap. |
Add ability to list available versions of requested package compatible with current Python interpreter. Add ability to query Python version prerequisites for a specific version of a package. Add ability to constrain package versions compatible with a Python version range. These are experimental features that use data directly from PyPi using the new JSON API. Add purity indicator of package upon successful installation. Relates https://warehouse.readthedocs.io/api-reference/json, #602, #610
Add package purity detection using Wheel format metadata as specified here https://www.python.org/dev/peps/pep-0491/#what-s-the-deal-with-purelib-vs-platlib. Add process output flag to run pip command in order to parse output of the pip install result before displaying it to the user. Add variant type to log pure / impure state. Drop os version dependency all together platform / architecture / (os) / python for all packages pure and impure. Only for pure packages - drop specific Python version and use only major part. Relates #602, #61
Fix the SchemaError "should be instance of" PackageRequest but is None. This is occuring when the requires list includes None and validation takes place.
Modify the allowed package name regex string to allow for packages that include a dash in their names which is quite common for Python packages. This eliminates issues like the one below: ERROR PackageRequestError: Not a valid package name: u'import-class'.
Fix the problem when the while loop was not checking for user choice being in the suitable version list.
Add missing parentheses to print function to comply with Python3 changes merged into master.
Some studios have an internal PyPI index (Nexus, JFrog, devpi) as such using the default endpoint is not ideal. Allow the user to specify which endpoint to use via the pip config file. The field required is `rez.pypi-api-endpoint`. Failing to detect an active config or if the key is missing will result in the default url being used. Relates: https://pypi.org/, https://pip.pypa.io/en/stable/user_guide/#configuration, https://pip.pypa.io/en/stable/reference/pip_config/
Description
[Depends on PR #602 #654 #656]
This PR has been extracted from #628 (a lot of the relevant discussion still lives there)
pip QOL improvements
To further ensure compatibility with current platform this PR adds some experimental features
such as package version listing, querying and Python version constraint based search. These features
use the most "reliable" source of package data being the PyPi via the new JSON API.
The reason these features are marked experimental is because currently there is no standard of how the metadata should be defined and sometimes it varies widely between packages. According to my tests it seems to be working for the most part but still a lot of things to consider.
Breakdown
ability to list all available versions of a package based on PyPi
ability to query Python version prerequisites of a specific version of a package based on PyPi
ability to retrieve and install a specific version of a Python package based on a Python version range (for example 3.6+)
Process
the PyPi JSON API
Type of change
Remarks
How Has This Been Tested?
In terms of the experimental features, although it does work fairly well it is a matter of how much trust
we can put into non-standardized metadata. For example a package might state Python version
requirement 3.5+ but might also work perfectly well with Python 2.7 or lower than Python 3.5...
Test Configuration:
Todos
Demos