executor: update setuptools/wheel first before other (parallel) installs

[sileht]

If a package is build from source because no pep517 build system are
indicated in pyproject.toml or the file is missing, pep517 will fallback
to setuptools and wheel installed within the virtualenv to build it.

eg:

https://github.com/pypa/pip/blob/22.3.1/src/pip/_vendor/pep517/wrappers.py#L167
https://github.com/pypa/pip/blob/22.3.1/src/pip/_vendor/pep517/wrappers.py#L220
https://github.com/pypa/pip/blob/22.3.1/src/pip/_vendor/pep517/wrappers.py#L273

Resolves: #7046

• Added tests for changed code.
• Updated documentation for changed code.

[radoering]

Why? Normally, setuptools in the target environment shouldn't be relevant because of

poetry/src/poetry/utils/pip.py

Line 38 in 71b3d6e

args.insert(1, "--use-pep517")

[sileht]

Why?

I added the missing issue in the description, eg: Resolves: #7046

[sileht]

What is the minimal version of pip poetry support?

Because it looks like the build-system fallback has been dropped in 23.0.

[dimbleby]

I still don't understand what the connection is between the installation of setuptools in the target environment, and the installation of setuptools in the temporary environment that pip uses to build (eg) forbiddenfruit as at #7046.

Surely they are completely different setuptools?

[sileht]

I still don't understand what the connection is between the installation of setuptools in the target environment, and the installation of setuptools in the temporary environment that pip uses to build (eg) forbiddenfruit as at #7046.

Surely they are completely different setuptools?

I continue to dig, and I don't get it either.

The interesting logs are:

 • Updating setuptools (63.2.0 -> 65.5.1)  -----> Update of setuptools inside the target venv.

I was misled by this, and as you seem to confirm, this is unrelated to the issue we see later in the logs.

Then pip fails:

Command ['/venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/venv', '--no-deps', '/root/.cache/pypoetry/artifacts/91/e3/3a/ee514b36115af9f8b73c52022cf789cee2e2d0603ada6f03311b316242/forbiddenfruit-0.1.4.tar.gz']' returned non-zero exit status 2.
...
Installing build dependencies: started
Installing build dependencies: finished with status 'done' <=========== HERE a new setuptools is supposed to be installed in the temporary pep517 venv
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
....
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/tmp/pip-build-env-k0jhdblz/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 8, in <module>
import _distutils_hack.override  # noqa: F401
ModuleNotFoundError: No module named '_distutils_hack.override' <======= HERE setuptools looks partially installed.

I'm going to close this PR.
This clearly looks like a pip issue.

[dimbleby]

I could just about believe an explanation that said that pip somehow re-uses the nearest setuptools when creating the isolated environment in which to peform the build.

However I've no reason to think that's true, it's pure speculation.

If you're hitting the issue frequently enough that this is worth your trouble, perhaps you can also test the change to see whether - experimentally - it seems to help or not.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.