package installation failure

[sileht]

• Poetry version: 1.2.2
• Python version: 3.10.8
• OS version and name: Ubuntu 20.04
• pyproject.toml: https://gist.github.com/sileht/1ea186354c1df26c4cbed623b6582157

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

poetry failed to install packages from time to time with the error below.
I suspect it only occurs with the package is not pep518 compatible and pip requires building a wheel.
I'm not sure if it's directly linked to poetry, or if this is just poetry that triggers a pip/setuptools bug?

One of my guess is, setuptools get updated while another package tries to build a wheel and since all setuptools files are replaced during the process, the links/files inside /tmp/pip-build-env-k0jhdblz/overlay/ may not exists for a short time.

It's not 100% reproductive, but it occurs a lot inside our GitHub workflows.

#39 18.26   • Installing urllib3 (1.26.12)
#39 19.62   • Installing asyncpg (0.27.0)
#39 19.62   • Installing beautifulsoup4 (4.11.1)
#39 19.62   • Installing cattrs (22.2.0)
#39 19.62   • Installing click (8.1.3)
#39 19.62   • Installing bytecode (0.13.0)
#39 19.62   • Installing cryptography (38.0.3)
#39 21.00   • Installing ddsketch (2.0.4)
#39 21.03   • Installing envier (0.4.0)
#39 21.03   • Installing fastapi (0.86.0)
#39 21.17   • Installing forbiddenfruit (0.1.4)
#39 21.47   • Installing greenlet (1.1.3.post0)
#39 21.94   • Installing h2 (4.1.0)
#39 22.40   • Installing httpcore (0.15.0)
#39 22.50   • Installing httptools (0.5.0)
#39 22.54   • Installing itsdangerous (2.1.2)
#39 22.89   • Installing jsonschema (4.17.0)
#39 23.34   • Installing markupsafe (2.1.1)
#39 23.83   • Installing passlib (1.7.4)
#39 23.87   • Installing python-dotenv (0.21.0)
#39 23.89   • Installing python-json-logger (2.0.4)
#39 24.40   • Installing pyyaml (6.0)
#39 24.61   • Installing redis (4.3.4)
#39 25.12   • Installing requests (2.28.1)
#39 25.13   • Installing rfc3986 (1.5.0)
#39 25.81   • Installing setproctitle (1.2.3)
#39 25.95   • Updating setuptools (63.2.0 -> 65.5.1)
#39 26.52   • Installing sqlalchemy (1.4.43)
#39 26.52   • Installing tenacity (8.1.0)
#39 26.53   • Installing uvloop (0.17.0)
#39 27.22   • Installing watchfiles (0.18.0)
#39 27.85 
#39 27.86   CalledProcessError
#39 27.86 
#39 27.86   Command '['/venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/venv', '--no-deps', '/root/.cache/pypoetry/artifacts/91/e3/3a/ee514b36115af9f8b73c52022cf789cee2e2d0603ada6f03311b316242/forbiddenfruit-0.1.4.tar.gz']' returned non-zero exit status 2.
#39 27.86 
#39 27.86   at /usr/local/lib/python3.10/subprocess.py:526 in run
#39 28.05        522│             # We don't call process.wait() as .__exit__ does that for us.
#39 28.05        523│             raise
#39 28.05        524│         retcode = process.poll()
#39 28.05        525│         if check and retcode:
#39 28.05     →  526│             raise CalledProcessError(retcode, process.args,
#39 28.05        527│                                      output=stdout, stderr=stderr)
#39 28.05        528│     return CompletedProcess(process.args, retcode, stdout, stderr)
#39 28.05        529│ 
#39 28.05        530│ 
#39 28.05 
#39 28.06 The following error occurred when trying to handle this error:
#39 28.06 
#39 28.06 
#39 28.06   EnvCommandError
#39 28.06 
#39 28.06   Command ['/venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/venv', '--no-deps', '/root/.cache/pypoetry/artifacts/91/e3/3a/ee514b36115af9f8b73c52022cf789cee2e2d0603ada6f03311b316242/forbiddenfruit-0.1.4.tar.gz'] errored with the following return code 2, and output: 
#39 28.06   Processing /root/.cache/pypoetry/artifacts/91/e3/3a/ee514b36115af9f8b73c52022cf789cee2e2d0603ada6f03311b316242/forbiddenfruit-0.1.4.tar.gz
#39 28.06     Installing build dependencies: started
#39 28.06     Installing build dependencies: finished with status 'done'
#39 28.06     Getting requirements to build wheel: started
#39 28.06     Getting requirements to build wheel: finished with status 'done'
#39 28.06   ERROR: Exception:
#39 28.06   Traceback (most recent call last):
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
#39 28.06       status = run_func(*args)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
#39 28.06       return func(self, options, args)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 369, in run
#39 28.06       requirement_set = resolver.resolve(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
#39 28.06       collected = self.factory.collect_root_requirements(root_reqs)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
#39 28.06       req = self._make_requirement_from_install_req(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
#39 28.06       cand = self._make_candidate_from_link(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
#39 28.06       self._link_candidate_cache[link] = LinkCandidate(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
#39 28.06       super().__init__(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
#39 28.06       self.dist = self._prepare()
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
#39 28.06       dist = self._prepare_distribution()
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
#39 28.06       return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 438, in prepare_linked_requirement
#39 28.06       return self._prepare_linked_requirement(req, parallel_builds)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 524, in _prepare_linked_requirement
#39 28.06       dist = _get_prepared_distribution(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
#39 28.06       abstract_dist.prepare_distribution_metadata(
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
#39 28.06       self._install_build_reqs(finder)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
#39 28.06       build_reqs = self._get_build_requires_wheel()
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
#39 28.06       return backend.get_requires_for_build_wheel()
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 685, in get_requires_for_build_wheel
#39 28.06       return super().get_requires_for_build_wheel(config_settings=cs)
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 172, in get_requires_for_build_wheel
#39 28.06       return self._call_hook('get_requires_for_build_wheel', {
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook
#39 28.06       raise BackendUnavailable(data.get('traceback', ''))
#39 28.06   pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
#39 28.06     File "/venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend
#39 28.06       obj = import_module(mod_path)
#39 28.06     File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
#39 28.06       return _bootstrap._gcd_import(name[level:], package, level)
#39 28.06     File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#39 28.06     File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#39 28.06     File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
#39 28.06     File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#39 28.06     File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#39 28.06     File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#39 28.06     File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
#39 28.06     File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
#39 28.06     File "<frozen importlib._bootstrap_external>", line 883, in exec_module
#39 28.06     File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#39 28.06     File "/tmp/pip-build-env-k0jhdblz/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 8, in <module>
#39 28.06       import _distutils_hack.override  # noqa: F401
#39 28.06   ModuleNotFoundError: No module named '_distutils_hack.override'
#39 28.06   
#39 28.06   
#39 28.06 
#39 28.06   at /usr/local/lib/python3.10/site-packages/poetry/utils/env.py:1476 in _run
#39 28.21       1472│                 output = subprocess.check_output(
#39 28.21       1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
#39 28.22       1474│                 )
#39 28.22       1475│         except CalledProcessError as e:
#39 28.22     → 1476│             raise EnvCommandError(e, input=input_)
#39 28.22       1477│ 
#39 28.22       1478│         return decode(output)
#39 28.22       1479│ 
#39 28.22       1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
#39 28.22 
#39 28.22 The following error occurred when trying to handle this error:
#39 28.22 
#39 28.22 
#39 28.22   PoetryException
#39 28.22 
#39 28.22   Failed to install /root/.cache/pypoetry/artifacts/91/e3/3a/ee514b36115af9f8b73c52022cf789cee2e2d0603ada6f03311b316242/forbiddenfruit-0.1.4.tar.gz
#39 28.22 
#39 28.22   at /usr/local/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
#39 28.23        47│ 
#39 28.23        48│     try:
#39 28.23        49│         return environment.run_pip(*args)
#39 28.23        50│     except EnvCommandError as e:
#39 28.23     →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
#39 28.23        52│ 
#39 28.23 
#39 28.23   • Installing websockets (10.4)
#39 28.23   • Installing xmltodict (0.13.0)
#39 ERROR: process "/bin/sh -c poetry install --no-dev --sync --no-root" did not complete successfully: exit code: 1
------
 > [python-builder  9/11] RUN poetry install --no-dev --sync --no-root:
#39 28.22   at /usr/local/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
#39 28.23        47│ 
#39 28.23        48│     try:
#39 28.23        49│         return environment.run_pip(*args)
#39 28.23        50│     except EnvCommandError as e:
#39 28.23     →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
#39 28.23        52│ 
#39 28.23 
#39 28.23   • Installing websockets (10.4)
#39 28.23   • Installing xmltodict (0.13.0)
------

[dimbleby]

if you're right, simplest workaround is to upgrade setuptools before poetry install.

cf #6582

[asaf-kali]

Still having this issue, python 3.10, poetry 1.4.2, Github CI.
Removing --sync flag from poetry install fixed the issue.