InvalidVersion exception when invalid version used on Setuptools 66

[jstucke]

setuptools version

66.0.0

Python version

3.8

OS

Ubuntu 20.04

Additional environment information

only happening when not running inside a venv

Description

Trying to install certain pip packages like e.g. ssdeep results in an pkg_resources.extern.packaging.version.InvalidVersion exception. This seems to be related to this bug where certain ubuntu/debian packages install python packages (in this case python3-distro-info and python-debian) with versions that don't conform to PEP 440

Expected behavior

With setuptools <66.0.0 this does not cause an error

How to Reproduce

1. docker run -it --rm --entrypoint=bash ubuntu:focal
2. apt update && apt install python3 python3-pip python3-distro-info python-debian libfuzzy-dev -y
3. python3 -m pip install -U setuptools pip wheel
4. python3 -m pip install ssdeep

Output

python3 -m pip install --user ssdeep --no-cache-dir
Collecting ssdeep
  Downloading ssdeep-3.4.tar.gz (110 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.8/110.8 kB 3.6 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [28 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-3dwjl5j0/ssdeep_f2bd2f0d9f384d9a92bf4f0f6f3090d3/setup.py", line 108, in <module>
          setup(
        File "/home/vagrant/.local/lib/python3.8/site-packages/setuptools/__init__.py", line 86, in setup
          _install_setup_requires(attrs)
        File "/home/vagrant/.local/lib/python3.8/site-packages/setuptools/__init__.py", line 80, in _install_setup_requires
          dist.fetch_build_eggs(dist.setup_requires)
        File "/home/vagrant/.local/lib/python3.8/site-packages/setuptools/dist.py", line 874, in fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 815, in resolve
          dist = self._resolve_dist(
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 844, in _resolve_dist
          env = Environment(self.entries)
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1044, in __init__
          self.scan(search_path)
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1077, in scan
          self.add(dist)
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1096, in add
          dists.sort(key=operator.attrgetter('hashcmp'), reverse=True)
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2631, in hashcmp
          self.parsed_version,
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2678, in parsed_version
          self._parsed_version = parse_version(self.version)
        File "/home/vagrant/.local/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__
          raise InvalidVersion(f"Invalid version: '{version}'")
      pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '0.23ubuntu1'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[nachovizzo]

I'm also facing this error

[jaraco]

This issue is by design (#2497). Non-conformant versions have been discouraged since ~2014, deprecated in packaging since late 2020 and deprecated in Setuptools since Oct 2021.

Projects should adapt to provide conformant versions (i.e. 0.23+ubuntu1) or environments that rely on such packages should pin to Setuptools < 66. These changes are necessary to keep Setuptools healthy and aligned with the goals of the packaging ecosystem.

Unfortunately, deprecation and removal is the main way I know to inform downstream consumers of breaking changes like these.

If the disruption is too great or the workarounds aren't suitable, this project may consider backing out the changes for another limited period, but only as a mitigation measure. Please first explore mitigation measures, such as pinning setuptools in the build-system.requires in the projects or work with the pip team to allow specifying constraints on build system requirements.

I'm also facing this error

Please use the thumbs-up on the original post to register that this issue also affects your project.

[yishaibeeri]

Quick note that in some cases internal dependencies several steps down the chain from the package I'm using are bringing in the new version of setuptools, making it hard to impossible to pin setuptools as a user who's running pip install. It also means install workflows that worked yesterday are broken today, even when pinning all my requirements.txt rows.
This means wide impact without an available workaround for users.
Am I missing an available workaround?
Please consider coordinating the rollout of this change with the pip team (et al) to allow for effective pinning

[karlicoss]

I'm getting this issue when I'm using setup.py with https://github.com/pypa/setuptools_scm
I have the following configuration:

        use_scm_version={
            'version_scheme': 'python-simplified-semver',
            'local_scheme': 'dirty-tag',
        },
        setup_requires=['setuptools_scm'],

With setuptools 66.0.0, I get

...
           self._parsed_version = parse_version(self.version)
        File "/usr/local/lib/python3.10/dist-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__
          raise InvalidVersion(f"Invalid version: '{version}'")
      pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '1.1build1'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

When I roll back the setuptools version to 65.7.0, it works as expected (e.g. my package version ends up as 1.0.20220516.dev56)

[jstucke]

Am I missing an available workaround?

If I am not mistaken, using a virtual environment should work (since all the packages with offending versions seem to come via apt or equivalent). If the package you are trying to install or its dependencies do not conform to PEP 440, this will of course not help.
Also uninstalling the system packages (python3-distro-info, python-debian, etc.) seems to work but they are probably needed by some other packages. python3-distro-info for example is used by update-manager-core, which is in turn used by update-manager, which is used by ubuntu-desktop, so it might not be feasible.

[kchomski]

I have the same issue with setuptools==65.5.1.
When tagging and trying to install a local project I get:

pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '1.23.0-test.1.2.3'

$ pip list | grep setuptools
setuptools                65.5.1

1.23.0-test.1.2.3 is a valid semver 2.0.0 tag.

EDIT: this happens inside venv (unlike OP)

$ which python
/c/USERS/foobar/.virtualenvs/venv-3.8/Scripts/python

EDIT 2:
Seems like an issue on my end - after checking PEP440 I learned that it's not fully compatible with Semantic Versioning:

Semantic versions containing a hyphen (pre-releases - clause 10) or a plus sign (builds - clause 11) are not compatible with this PEP and are not permitted in the public version field.

[jstucke]

Since the error message is somewhat cryptic (it would help if it also printed out the offending package in addition to the version), I dug around a bit to help match the version from the error message to the package that caused them:

• python-debian (is probably irrelevant because Python2.7 only)
  • 0.18ubuntu0.18.04.1 bionic
  • 0.1.36ubuntu1 focal
• python3-distro-info
  • 0.23ubuntu1 focal
  • 1.1build1 jammy
  • 1.1ubuntu1 kinetic
• python3-reportbug
  • 7.5.3~deb10u1 buster
  • 7.1.8ubuntu1 bionic
  • 7.6.0ubuntu1 focal
  • 11.4.1ubuntu1 jammy
  • 11.5.0ubuntu1 kinetic/lunar

[Greg7000]

When I am in a context where I face this error and still want a custom tag. I use the '+' sign (which you can see from jaraco comment.**

For example, lets say I want mycode-1.0<CUSTOM_TAG>

Assuming I want my custom tag to be win_bob

I set my version to mycode-1.0+win_bob

Which is then interpreted as mycode-1.0+win.bob. For my needs, this is acceptable.

Check https://peps.python.org/pep-0440/#local-version-segments for more information.

[RyanPaulMcKenna]

Switching to setuptools 65 solved it for me, now I have new error.
pip install setuptools==65

[cendrillon]

Can we make a list of libraries like this that break their clients and then cause breakages everywhere in the python ecosystem? There's no regard for the end user, I think a list would be good so that the libraries I use wont break when setuptools decides to become pedantic about something.

[prathik8794]

even after downgrading the version of setuptools im still getting version exception, any suggestions.

[richard-scott]

Same here. The issue has shown up in my codebase in the last 3 days, and I have made no changes other than updating to the latest packages available in Pip. I update my packages before every build, so I would have been using the latest packages 4 days ago, and things were fine then.

[amgcc]

I too cannot make this work with a downgraded setuptools. My usecase is trying to run pip install -e . on a conda environment that includes the mosek package. I am now forced to upgrade the package which in turn requires a python upgrade which is quite an inconvenience.

Running command: <envs/2eba25fea3f21c28f9a5a7fb89eaa5230f442014/bin/python -m pip install -e .>                                                                            
Obtaining file:///......
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
  Getting requirements to build editable: started
  Getting requirements to build editable: finished with status 'done'
  Preparing editable metadata (pyproject.toml): started
  Preparing editable metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: my_package
  Building editable for my_package (pyproject.toml): started
  Building editable for my_package (pyproject.toml): finished with status 'done'
  Created wheel for my_package: filename=my_package-0.0-0.editable-py3-none-any.whl size=1083 sha256=9b7a2cd17e1223e5590c5cb69438711cf5b79ad88f36b2fed035e66bdde7a8d4
  Stored in directory: /...
Successfully built my_package
WARNING: Error parsing dependencies of mosek: Invalid version: '-version-'

[richard-scott]

This is back again with setuptools 75.3.0:

File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 376, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 319, in run_setup
          exec(code, locals())
        File "<string>", line 127, in <module>
        File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
        File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 145, in setup
          _setup_distribution = dist = klass(attrs)
        File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 337, in __init__
          self.metadata.version = self._normalize_version(self.metadata.version)
        File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 373, in _normalize_version
          normalized = str(Version(version))
        File "/private/var/folders/xt/7t55qbzj5hd40blg_mk_tq_m0000gn/T/pip-build-env-fiwf2_oc/overlay/lib/python3.9/site-packages/setuptools/_vendor/packaging/version.py", line 202, in __init__
          raise InvalidVersion(f"Invalid version: '{version}'")
      packaging.version.InvalidVersion: Invalid version: '§5.0.1'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

[prathik8794]

I resolved my issue, the problem was due to pip, i had to set pip to 20.3.3, im using python 3.8, setuptools 65.7.0 anything less than 66.

[richard-scott]

I resolved my issue, the problem was due to pip, i had to set pip to 20.3.3, im using python 3.8, setuptools 65.7.0 anything less than 66.

Thanks, but that didn't help me. I've tried pip==20.3.3, setuptools=58.0.4 and it's still broken.