Run type-annotation linters in CI environment

[happz]

The goal si to verify the code and its annotations are compatible with Python packages shipped as system packages.

Pull Request Checklist

• implement the feature
• write the documentation
• extend the test coverage
• update the specification
• adjust plugin docstring
• modify the json schema
• mention the version
• include a release note

[LecrisUT]

Shouldn't we just be using dnf install python3-mypy so that we are actually using the systems packages here? I see in the test logs that it is still downloading from pypi the packages.

[LecrisUT]

Oh, you're only filtering types-* there. I was looking at urllib3-2.2.1 still being downloaded.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
requests 2.28.2 requires urllib3<1.27,>=1.21.1, but you have urllib3 2.2.1 which is incompatible.

Looking more on the issue of urllib3, that seems to be quite a complicated can of worms.

[happz]

Shouldn't we just be using dnf install python3-mypy so that we are actually using the systems packages here? I see in the test logs that it is still downloading from pypi the packages.

I wouldn't think so, because mypy (and pyright) are not tmt requirements, but linters. The idea is to use system dependencies, but defined linters looking at the project from outside. With system mypy, we might be using older versions often, which may not recognize issues newer versions were taught to report.

Oh, you're only filtering types-* there. I was looking at urllib3-2.2.1 still being downloaded.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
requests 2.28.2 requires urllib3<1.27,>=1.21.1, but you have urllib3 2.2.1 which is incompatible.

Looking more on the issue of urllib3, that seems to be quite a complicated can of worms.

Yeah, it seems to be pulled in because of types-requests. TBH, I did not even check the log of pip install, and yeah, this is something I'd like to avoid. No idea why types-requests feels the need to pull in a genuine package, maybe because urllib3 does not ship type stubs separately, but an action like this would ruing the idea of "testing against system packages" :/

I think we'll get there, but it is indeed a bigger can of worms than it seemed last week :)

[LecrisUT]

I wouldn't think so, because mypy (and pyright) are not tmt requirements, but linters. The idea is to use system dependencies, but defined linters looking at the project from outside. With system mypy, we might be using older versions often, which may not recognize issues newer versions were taught to report.

Yeah, fully agree. Was more concerned about the actual packages, but then I saw that you are only using types-*, which seems a-ok. But I think the version of the types-* needs to be matched with the version found in the system's pip, i.e. search the pre-commit-config.yaml (or maybe other approach later) for the types-*packages, search pip list or such for the actual package version, and then write the version constrain to match the actual package

No idea why types-requests feels the need to pull in a genuine package, maybe because urllib3 does not ship type stubs separately, but an action like this would ruing the idea of "testing against system packages" :/

Found out. It's because urllib3 >= 2.0 already includes typing, and the types-request constraint used, does not apply in F40 case, thus getting the latest version types-request which uses the latest urllib3 version. The workflow above should take care of that though, WDYT?

[LecrisUT]

How about bumping some of the minimum requirements?

• jinja2 >= 3.0: can get rid of types-jinja2
• types-jsonschema might not be necessary anymore? Seems like the simple typing are merged (primary issue), and it's mainly quite advanced typing that we most likely will not use that is still open.
• types-urllib3 can be removed once urllib3 >= 2.0 is landed. Epel9 might be tricky though.
• types-babel can be removed from 2.12.1. Didn't find what the version is on Epel9, F39 can remove already
• types-setuptools 🤷 what is this one used at?
• types-docutils, types-Markdown, types-requests still needed for the foreseeable future.

But does mypy not have a more to automatically download the missing typings from typeshed?

[happz]

How about bumping some of the minimum requirements?

As long as they can be satisfied with RPMs available to us, I don't see a problem in general. I'm not sure what would happen if we bump a package version too much, rpmbuild would fail?

• jinja2 >= 3.0: can get rid of types-jinja2
• types-jsonschema might not be necessary anymore? Seems like the simple typing are merged (primary issue), and it's mainly quite advanced typing that we most likely will not use that is still open.
• types-urllib3 can be removed once urllib3 >= 2.0 is landed. Epel9 might be tricky though.
• types-babel can be removed from 2.12.1. Didn't find what the version is on Epel9, F39 can remove already
• types-setuptools 🤷 what is this one used at?

Maybe something related to reousrce_files, importlib? I don't recall we'd use packaging-related functionality in tmt code for anything beyond that.

• types-docutils, types-Markdown, types-requests still needed for the foreseeable future.

But does mypy not have a more to automatically download the missing typings from typeshed?

It does download missing types, with --install-types, on the other hand it seemed better to list them explicitly and make sure they are present in mypy and pyright environments rather than leave it on linter guessing.

[LecrisUT]

• types-setuptools 🤷 what is this one used at?

Maybe something related to reousrce_files, importlib? I don't recall we'd use packaging-related functionality in tmt code for anything beyond that.

types-setuptools seems to only cover pkg_resources and distutils, both of which tmt seems to have already upgraded. That's why I was asking.

But does mypy not have a more to automatically download the missing typings from typeshed?

It does download missing types, with --install-types, on the other hand it seemed better to list them explicitly and make sure they are present in mypy and pyright environments rather than leave it on linter guessing.

Good to keep this option in mind. If we end up decommissioning the pre-commit in favor of this workflow, that might be necessary to get the typeshed. At the very least it is worth making a temp commit running it with --install-types to see if it uses the restrictions as described in the previous comment

Edit: Did a bit of navigation around install_types and it seems it does not include the version constraint ¹, and the typeshed packages are not version matched to the original package (the intent upstream there is that the typeshed have no executable code, including the original package). So some kind of logic to search for the missing stubs and manually create the requirements.txt might be needed.

Footnotes

1. https://github.com/python/mypy/blob/43a605f742bd554acbdff9bea74c764621e3aa44/mypy/build.py#L2811 ↩

[happz]

/packit test

[martinhoyer]

This was fixed on rhel-9 btw. Do we need to keep compatibility with older releases, or can we get rid of the workaround completely?

EDIT: actually, not fixed yet. "Release pending" :)

[martinhoyer]

I wonder if we can use a single source of truth for the list of the packages. Maybe something similar to the existing make develop perhaps?

[happz]

I guess we could use make develop directly. However, it's a bit unsatisfactory solution, instead of a declaration of packages required by a test, we'd instead take action and run make. I'm going to switch to this make develop command for now, but maybe we need to save these requirements somewhere from which both test and make could read it.

[happz]

Changed in dcd0402