Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Follow most recent packaging specification for wheel name normalization #394

Conversation

mkniewallner
Copy link
Member

Highlighted by python-poetry/poetry#5782 and looking into PyPA packaging specification that supersedes PEP 427:

In distribution names, any run of -_. characters (HYPHEN-MINUS, LOW LINE and FULL STOP) should be replaced with _ (LOW LINE), and uppercase characters should be replaced with corresponding lowercase ones. This is equivalent to PEP 503 normalisation followed by replacing - with _. Tools consuming wheels must be prepared to accept . (FULL STOP) and uppercase letters, however, as these were allowed by an earlier version of this specification.

So it looks like the regex we use to escape wheel name should be updated to what PEP 503 defines, with the minor change of using _ for the replacement character instead of -.

  • Added tests for changed code.
  • Updated documentation for changed code.

@mkniewallner
Copy link
Member Author

Keeping this as a draft for now, since I'm not entirely sure about the implications of changing the logic here.
If anyone has more insights, I would gladly take your feedback.

@neersighted neersighted requested a review from abn June 6, 2022 17:52
neersighted
neersighted previously approved these changes Jun 6, 2022
Copy link
Member

@neersighted neersighted left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me as this only affects generated artifacts (bdists) and the spec explicitly calls out that tools must continue to support wheel names generated using the old logic -- given this is a subset of possible names before the spec update, it should be fully compatible. Still, going to wait for @abn to concur.

@mkniewallner mkniewallner marked this pull request as ready for review June 6, 2022 18:01
@abn
Copy link
Member

abn commented Jun 6, 2022

As this is only affecting generated bdist files, I agree with @neersighted.

However (outside the scope of this PR), I am now curious if we do the right thing in cases where PEP 503 normalisation does not match wheel name escapes - ie. how do we tell if foo-bar in a package source is the same as foo_bar.whl direct reference dependency?

@mkniewallner
Copy link
Member Author

As this is only affecting generated bdist files, I agree with @neersighted.

Thanks to both of you for confirming that. I also think that since this only touches the generation of the wheel name (not it's retrieval locally nor on a distant remote, AFAIK), we should be safe here.

However (outside the scope of this PR), I am now curious if we do the right thing in cases where PEP 503 normalisation does not match wheel name escapes - ie. how do we tell if foo-bar in a package source is the same as foo_bar.whl direct reference dependency?

That's a valid concern. I can take a look at what we currently do later on, since the normalisation of wheel names may occur in various different situations (local cache, distant remote, etc.), so definitely worth a look.

@@ -29,5 +29,5 @@ def escape_version(version: str) -> str:


def escape_name(name: str) -> str:
"""Escaped wheel name as specified in :pep:`427#escaping-and-unicode`."""
return re.sub(r"[^\w\d.]+", "_", name, flags=re.UNICODE)
"""Escaped wheel name as specified in https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode."""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe expand this doc string to indicate that this should only be used for generation of wheels never to canonicalize artifact package names.

@mkniewallner mkniewallner force-pushed the follow-most-recent-packaging-spec-wheel-name branch from ed7078e to 0651747 Compare June 7, 2022 18:07
@mkniewallner mkniewallner requested a review from abn June 7, 2022 18:41
Co-authored-by: Bjorn Neergaard <bjorn@neersighted.com>
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jun 7, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@mkniewallner mkniewallner requested a review from neersighted June 7, 2022 20:10
@neersighted neersighted merged commit dfd8e4b into python-poetry:main Jun 7, 2022
@mkniewallner mkniewallner deleted the follow-most-recent-packaging-spec-wheel-name branch June 7, 2022 22:21
@radoering radoering mentioned this pull request Jul 9, 2022
bostonrwalker pushed a commit to bostonrwalker/poetry-core that referenced this pull request Aug 29, 2022
…on (python-poetry#394)

* test(masonry): add tests for `escape_*` methods
* feat(masonry): follow newer PyPA specification for wheel name

As per https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format,
the specification replaces the original PEP 427 specification.
The specification for the distribution name is https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode.

* refactor(masonry): indicate that `escape_name` is for generated wheels only
* doc(masonry): improve `escape_name` documentation

Co-authored-by: Bjorn Neergaard <bjorn@neersighted.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants