Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the py.typed marker file usage #4172

Merged
merged 8 commits into from
Sep 30, 2024
Merged

Conversation

kjagiello
Copy link
Contributor

@kjagiello kjagiello commented Sep 1, 2024

Description

As defined in PEP 561, the py.typed marker files for namespace packages should be placed in the submodules of the namespace. In commit 732ea8a, the markers were added to the subpackage roots, which serves no purpose. This changeset addresses all marker-related issues in this repository.

A summary of the changes:

  • Removed the unnecessary py.typed markers from the subpackage roots.
  • Added the missing marker to the prometheus exporter. Fixes py.typed not included in wheels #4113.
  • Added the missing marker to opentelemetry._events.
  • Removed the accidentally reintroduced jaeger exporters, which were originally removed in commit 1625b35.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • tox -e mypy
  • Installed the packages from this commit in a project of mine and verified that pyright was satisfied with the code interacting with opentelemetry-api, opentelemetry-sdk and opentelemetry-exporter-prometheus. Previously pyright complained about the exporter missing the marker.

Does This PR Require a Contrib Repo Change?

  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated

Copy link

linux-foundation-easycla bot commented Sep 1, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

As defined in PEP 561, the py.typed marker files for namespace packages
should be placed "in the submodules of the namespace". In commit
732ea8a, the markers were added to the subpackage roots, which serves no
purpose. This changeset addresses all marker-related issues in this
repository.

A summary of the changes:

- Removed the unnecessary py.typed markers from the subpackage roots.
- Added the missing marker to the prometheus exporter. Fixes open-telemetry#4113.
- Added the missing marker to opentelemetry._events.
- Removed the accidentally reintroduced jaeger exporters, which were
  originally removed in commit 1625b35.
@kjagiello kjagiello marked this pull request as ready for review September 1, 2024 18:34
@kjagiello kjagiello requested a review from a team September 1, 2024 18:34
@xrmx
Copy link
Contributor

xrmx commented Sep 3, 2024

Aren't we doing the following though?
For namespace packages (see [PEP 420](https://peps.python.org/pep-0420/)), the py.typed file should be in the submodules of the namespace, to avoid conflicts and for clarity.

@kjagiello
Copy link
Contributor Author

kjagiello commented Sep 3, 2024

Yes, you are following it for the most part. Most of the namespace subpackages have the marker, but there were some exceptions: the prometheus exporter and opentelemetry._events. The rest of this PR removes py.typed files that were not part of the submodules of the namespace, but instead were located in the package roots.

Perhaps an example will help to clarify my point:

  1. exporter/opentelemetry-exporter-otlp-proto-common/py.typed is on the package root level. This is incorrect and serves no purpose.
  2. exporter/opentelemetry-exporter-otlp-proto-common/src/opentelemetry/exporter/otlp/proto/common/py.typed lives in a submodule of the namepace. This is correct.

So this PR aims to get rid of the py.typed markers that are in incorrect places (see 1) and make sure that all the submodules of the namespace contain the marker (see 2).

@xrmx
Copy link
Contributor

xrmx commented Sep 3, 2024

@kjagiello thanks for the example. I have a followup question: why isn't opentelemetry-api/src/opentelemetry/py.typed enough to cover the rest of the modules in opentelemetry-api package though?

@xrmx
Copy link
Contributor

xrmx commented Sep 3, 2024

BTW any chance you can add a test calling pytight to avoid regressions and address #4086?

@xrmx
Copy link
Contributor

xrmx commented Sep 12, 2024

I think this is missing a py.typed for recently added sdk/_events

@kjagiello
Copy link
Contributor Author

kjagiello commented Sep 14, 2024

@kjagiello thanks for the example. I have a followup question: why isn't opentelemetry-api/src/opentelemetry/py.typed enough to cover the rest of the modules in opentelemetry-api package though?

One thing that comes to mind is that is that there should probably not be any py.typed at the top level. This connects to the mention of ambiguity in PEP 561. When the subpackages are installed, on the filesystem, they end up merged in the same package folder (in this case site-packages/opentelemetry/). Should a top-level py.typed from one subpackage signal that another subpackage is typed? I can see how a static type checker might not want a top-level py.typed to cover any of the submodules of a namespace.

Beware that this is just my interpretation of the PEP, but I think it kind of makes sense. The PEP is at least clear on having the py.typed markers in the submodules of the namespace.

BTW any chance you can add a test calling pyright to avoid regressions and address #4086?

Sure, I can give it a try!

@kjagiello kjagiello requested a review from a team as a code owner September 21, 2024 19:13
@kjagiello
Copy link
Contributor Author

To not expand the scope of this PR way too much, I did not address #4086 fully. Currently, pyright is configured to do only basic checks on opentelemetry-api, opentelemetry-sdk and opentelemetry-semantic-conventions.I have bumped reportMissingTypeStubs to error, which helped me find some more submodules either being unexpectedly implicit (missing __init__.py) or missing the py.typed marker.

I also have noticed that opentelemetry-sdk unnecessarily put py.typed in the submodules of the namespace root submodule. In this case I could replace all the marker occurrences in opentelemetry-sdk/src/opentelemetry/sdk/*/py.typed with a single marker in opentelemetry-sdk/src/opentelemetry/sdk/py.typed.

The exporters are currently not covered by pyright.

@kjagiello
Copy link
Contributor Author

Oops, had no idea that the workflows were generated using tox. Pushed a fix after running tox -e generate-workflows.

@ocelotl ocelotl merged commit fd169a7 into open-telemetry:main Sep 30, 2024
378 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

py.typed not included in wheels
4 participants