[BUG] Tests failing on PyPy on macOS

[jaraco]

When restoring the _distutils.msvc*compiler modules in #3505, the tests started failing with strange errors from pathlib:

______ TestBuildMetaBackend.test_editable_with_global_option_still_works _______
[gw1] darwin -- Python 3.7.13 /Users/runner/work/setuptools/setuptools/.tox/python/bin/python
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 140, in run
    self._create_wheel_file(bdist_wheel)
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 330, in _create_wheel_file
    files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 260, in _run_build_commands
    self._configure_build(dist_name, unpacked_wheel, build_lib, tmp_dir)
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 225, in _configure_build
    build_py.existing_egg_info_dir = self._find_egg_info_dir()
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 180, in _find_egg_info_dir
    return next(candidates, None)
  File "/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py", line 1125, in glob
    for p in selector.select_from(self):
  File "/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py", line 526, in _select_from
    entries = list(scandir_it)
OSError: [Errno 22] Invalid argument: '_meta/'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/runner/work/setuptools/setuptools/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/Users/runner/work/setuptools/setuptools/setuptools/_distutils/dist.py", line 973, in run_commands
    self.run_command(cmd)
  File "/Users/runner/work/setuptools/setuptools/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/Users/runner/work/setuptools/setuptools/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 154, in run
    raise errors.InternalError(cleandoc(msg)) from ex
distutils.errors.DistutilsInternalError: Support for editable installs via PEP 660 was recently introduced
in `setuptools`. If you are seeing this error, please report to:

https://github.com/pypa/setuptools/issues

Meanwhile you can try the legacy behavior by setting an
environment variable and trying to install again:

SETUPTOOLS_ENABLE_FEATURES="legacy-editable"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/Users/runner/work/setuptools/setuptools/setuptools/tests/test_build_meta.py", line 92, in __call__
    return getattr(backend, name)(*args, **kw)
  File "/Users/runner/work/setuptools/setuptools/setuptools/build_meta.py", line 441, in build_editable
    cmd, ".whl", wheel_directory, config_settings
  File "/Users/runner/work/setuptools/setuptools/setuptools/build_meta.py", line 397, in _build_with_temp_dir
    self.run_setup()
  File "/Users/runner/work/setuptools/setuptools/setuptools/build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 1, in <module>
  File "/Users/runner/work/setuptools/setuptools/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/Users/runner/work/setuptools/setuptools/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/Users/runner/work/setuptools/setuptools/setuptools/_distutils/core.py", line 215, in run_commands
    raise SystemExit("error: " + str(msg))
SystemExit: error: Support for editable installs via PEP 660 was recently introduced
in `setuptools`. If you are seeing this error, please report to:

https://github.com/pypa/setuptools/issues

Meanwhile you can try the legacy behavior by setting an
environment variable and trying to install again:

SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
"""

The above exception was the direct cause of the following exception:

self = <setuptools.tests.test_build_meta.TestBuildMetaBackend object at 0x00007f834ab13280>
tmpdir_cwd = local('/Users/runner/work/setuptools/setuptools')

    @pytest.mark.filterwarnings("ignore::setuptools.SetuptoolsDeprecationWarning")
    # Since the backend is running via a process pool, in some operating systems
    # we may have problems to make assertions based on warnings/stdout/stderr...
    # So the best is to ignore them for the time being.
    def test_editable_with_global_option_still_works(self, tmpdir_cwd):
        """The usage of --global-option is now discouraged in favour of --build-option.
        This is required to make more sense of the provided scape hatch and align with
        previous pip behaviour. See pypa/setuptools#1928.
        """
        path.build({**self._simple_pyproject_example, '_meta': {}})
        build_backend = self.get_build_backend()
        assert not Path("build").exists()
    
        cfg = {"--global-option": ["--mode", "strict"]}
        build_backend.prepare_metadata_for_build_editable("_meta", cfg)
>       build_backend.build_editable("temp", cfg, "_meta")

/Users/runner/work/setuptools/setuptools/setuptools/tests/test_build_meta.py:665: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Users/runner/work/setuptools/setuptools/setuptools/tests/test_build_meta.py:55: in method
    return self.pool.submit(caller, name, *args, **kw).result(TIMEOUT)
/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/concurrent/futures/_base.py:435: in result
    return self.__get_result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Future at 0x7f834e425910 state=finished raised SystemExit>

    def __get_result(self):
        if self._exception:
>           raise self._exception
E           SystemExit: error: Support for editable installs via PEP 660 was recently introduced
E           in `setuptools`. If you are seeing this error, please report to:
E           
E           https://github.com/pypa/setuptools/issues
E           
E           Meanwhile you can try the legacy behavior by setting an
E           environment variable and trying to install again:
E           
E           SETUPTOOLS_ENABLE_FEATURES="legacy-editable"

/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/concurrent/futures/_base.py:384: SystemExit
----------------------------- Captured stdout call -----------------------------
running dist_info
creating _meta/proj.egg-info
writing _meta/proj.egg-info/PKG-INFO
writing dependency_links to _meta/proj.egg-info/dependency_links.txt
writing top-level names to _meta/proj.egg-info/top_level.txt
writing _meta/proj.egg-info/PKG-INFO
writing dependency_links to _meta/proj.egg-info/dependency_links.txt
writing _meta/proj.egg-info/PKG-INFO
writing dependency_links to _meta/proj.egg-info/dependency_links.txt
writing top-level names to _meta/proj.egg-info/top_level.txt
writing manifest file '_meta/proj.egg-info/SOURCES.txt'
reading manifest file '_meta/proj.egg-info/SOURCES.txt'
writing manifest file '_meta/proj.egg-info/SOURCES.txt'
creating '/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw1/test_editable_with_global_opti0/_meta/proj-42.dist-info'
running editable_wheel
creating _meta/proj-42.dist-info/WHEEL
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 140, in run
    self._create_wheel_file(bdist_wheel)
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 330, in _create_wheel_file
    files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 260, in _run_build_commands
    self._configure_build(dist_name, unpacked_wheel, build_lib, tmp_dir)
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 225, in _configure_build
    build_py.existing_egg_info_dir = self._find_egg_info_dir()
  File "/Users/runner/work/setuptools/setuptools/setuptools/command/editable_wheel.py", line 180, in _find_egg_info_dir
    return next(candidates, None)
  File "/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py", line 1125, in glob
    for p in selector.select_from(self):
  File "/Users/runner/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py", line 526, in _select_from
    entries = list(scandir_it)
OSError: [Errno 22] Invalid argument: '_meta/'

It's not at all obvious to me what's going on here. What's an Invalid argument to pathlib.glob()? The error isn't consistent. Here it is passing.

In another run, it failed in a different test with a different path, but a similar error:

_ TestWheelCompatibility.test_dist_info_is_the_same_as_in_wheel[.post-[egg_info]\ntag_build = post\n-0.42.13-my.proj] _
[gw0] darwin -- Python 3.7.13 /Users/runner/work/setuptools/setuptools/.tox/python/bin/python

self = <setuptools.tests.test_dist_info.TestWheelCompatibility object at 0x00007fbbc57874e8>
name = 'my.proj', version = '0.42.13'
tmp_path = PosixPath('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw0/test_dist_info_is_the_same_as_6')
suffix = '.post', cfg = '[egg_info]\ntag_build = post\n'

    @pytest.mark.parametrize("name", "my-proj my_proj my.proj My.Proj".split())
    @pytest.mark.parametrize("version", ["0.42.13"])
    @pytest.mark.parametrize("suffix, cfg", EGG_INFO_OPTS)
    def test_dist_info_is_the_same_as_in_wheel(
        self, name, version, tmp_path, suffix, cfg
    ):
        config = self.SETUPCFG.format(name=name, version=version) + cfg
    
        for i in "dir_wheel", "dir_dist":
            (tmp_path / i).mkdir()
            (tmp_path / i / "setup.cfg").write_text(config, encoding="utf-8")
    
        run_command("bdist_wheel", cwd=tmp_path / "dir_wheel")
        wheel = next(tmp_path.glob("dir_wheel/dist/*.whl"))
        unpack_archive(wheel, tmp_path / "unpack")
>       wheel_dist_info = next(tmp_path.glob("unpack/*.dist-info"))

setuptools/tests/test_dist_info.py:181: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py:1125: in glob
    for p in selector.select_from(self):
../../../hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py:511: in _select_from
    for p in self.successor._select_from(path, is_dir, exists, scandir):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pathlib._WildcardSelector object at 0x00007fbbc7f6ce90>
parent_path = PosixPath('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw0/test_dist_info_is_the_same_as_6/unpack')
is_dir = <function Path.is_dir at 0x00007fbbc0228fc0>
exists = <function Path.exists at 0x00007fbbc0228f20>
scandir = <built-in function scandir>

    def _select_from(self, parent_path, is_dir, exists, scandir):
        try:
            with scandir(parent_path) as scandir_it:
>               entries = list(scandir_it)
E               OSError: [Errno 22] Invalid argument: '/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw0/test_dist_info_is_the_same_as_6/unpack/'

../../../hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/pathlib.py:526: OSError

Is the host running out of resources and thus failing spuriously?

[jaraco]

In other failed actions, I see tests failing on the same commit:

And there, the failure is yet something completely different.

____________________ test_pip_upgrade_from_source[pip<20.1] ____________________
[gw2] darwin -- Python 3.10.6 /Users/runner/work/setuptools/setuptools/.tox/python/bin/python

pip_version = 'pip<20.1'
venv_without_setuptools = <setuptools.tests.environment.VirtualEnv object at 0x1094509d0>
setuptools_wheel = PosixPath('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/wheel_build/setuptools-65.0.2.post20220817-py3-none-any.whl')
setuptools_sdist = PosixPath('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/sdist_build/setuptools-65.0.2.post20220817.tar.gz')

    @pytest.mark.skipif(
        'platform.python_implementation() == "PyPy"',
        reason="https://github.com/pypa/setuptools/pull/2865#issuecomment-965834995",
    )
    @pytest.mark.skipif(not access_pypi(), reason="no network")
    # ^-- Even when it is not necessary to install a different version of `pip`
    #     the build process will still try to download `wheel`, see #3147 and #2986.
    @pytest.mark.parametrize(
        'pip_version',
        [
            None,
            pytest.param('pip<20', marks=pytest.mark.xfail(reason='pypa/pip#6599')),
            'pip<20.1',
            'pip<21',
            'pip<22',
            pytest.param(
                'https://github.com/pypa/pip/archive/main.zip',
                marks=pytest.mark.xfail(reason='#2975'),
            ),
        ]
    )
    def test_pip_upgrade_from_source(pip_version, venv_without_setuptools,
                                     setuptools_wheel, setuptools_sdist):
        """
        Check pip can upgrade setuptools from source.
        """
        # Install pip/wheel, in a venv without setuptools (as it
        # should not be needed for bootstraping from source)
        venv = venv_without_setuptools
        venv.run(["pip", "install", "-U", "wheel"])
        if pip_version is not None:
>           venv.run(["python", "-m", "pip", "install", "-U", pip_version, "--retries=1"])

setuptools/tests/test_virtualenv.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
setuptools/tests/environment.py:34: in run
    return subprocess.check_output(cmd, *args, **kwargs)
../../../hostedtoolcache/Python/3.10.6/x64/lib/python3.10/subprocess.py:420: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = ([Path('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw2/test_pip_upgrade_from_source_p1/venv_without_setuptools/.env/bin/python'), '-m', 'pip', 'install', '-U', 'pip<20.1', ...],)
kwargs = {'cwd': Path('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw2/test_pip_up...sers/runner/work/setuptools/setuptools/.coverage', 'COV_CORE_SOURCE': ':', 'HOME': '/Users/runner', ...}, 'stdout': -1}
process = <Popen: returncode: 1 args: [Path('/private/var/folders/24/8k48jl6d249_n_qfx...>
stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '[Path('/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw2/test_pip_upgrade_from_source_p1/venv_without_setuptools/.env/bin/python'), '-m', 'pip', 'install', '-U', 'pip<20.1', '--retries=1']' returned non-zero exit status 1.

../../../hostedtoolcache/Python/3.10.6/x64/lib/python3.10/subprocess.py:524: CalledProcessError
---------------------------- Captured stdout setup -----------------------------
created virtual environment CPython3.10.6.final.0-64 in 584ms
  creator CPython3Posix(dest=/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw2/test_pip_upgrade_from_source_p1/venv/.env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, via=copy, app_data_dir=/Users/runner/Library/Application Support/virtualenv)
    added seed packages: pip==22.2.2, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Processing /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/wheel_build/setuptools-65.0.2.post20220817-py3-none-any.whl
Installing collected packages: setuptools
Successfully installed setuptools-65.0.2.post20220817
created virtual environment CPython3.10.6.final.0-64 in 752ms
  creator CPython3Posix(dest=/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pytest-of-runner/pytest-0/popen-gw2/test_pip_upgrade_from_source_p1/venv_without_setuptools/.env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, via=copy, app_data_dir=/Users/runner/Library/Application Support/virtualenv)
    added seed packages: pip==22.2.2, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
----------------------------- Captured stderr call -----------------------------
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10ae8cdf0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/pip/
ERROR: Could not find a version that satisfies the requirement pip<20.1 (from versions: none)
ERROR: No matching distribution found for pip<20.1

That one looks like maybe a spurious error due to network availability.

The tests run fine on my local macOS machine, although I can't test against PyPy due to incompatibility with my architecture.

[jaraco]

Tests have started passing again. No idea what was happening: