pytest-relaxed causes pytest-xdist testing to fail

[mtelka]

I tried to run tests for pytest-xdist with pytest-relaxed installed and disabled using -p no:relaxed set in PYTEST_ADDOPTS environment variable. I noticed that with this setup the pytest-xdist testing fails:

_________________ TestDistribution.test_dist_tests_with_crash __________________

self = <acceptance_test.TestDistribution object at 0x7fffadc69040>
pytester = <Pytester PosixPath('/tmp/pytest-of-marcel/pytest-150/test_dist_tests_with_crash0')>

    def test_dist_tests_with_crash(self, pytester: pytest.Pytester) -> None:
        if not hasattr(os, "kill"):
            pytest.skip("no os.kill")

        p1 = pytester.makepyfile(
            """
                import pytest
                def test_fail0():
                    assert 0
                def test_fail1():
                    raise ValueError()
                def test_ok():
                    pass
                def test_skip():
                    pytest.skip("hello")
                def test_crash():
                    import time
                    import os
                    time.sleep(0.5)
                    os.kill(os.getpid(), 15)
            """
        )
        result = pytester.runpytest(p1, "-v", "-d", "-n1")
>       result.stdout.fnmatch_lines(
            [
                "*Python*",
                "*PASS**test_ok*",
                "*node*down*",
                "*3 failed, 1 passed, 1 skipped*",
            ]
        )
E       Failed: nomatch: '*Python*'
E           and: 'created: 1/1 worker'
E           and: '============================= test session starts =============================='
E       fnmatch: '*Python*'
E          with: 'platform sunos5 -- Python 3.9.16, pytest-7.4.3, pluggy-1.3.0 -- $(BUILD_DIR)/.tox/py39/bin/python'
E       nomatch: '*PASS**test_ok*'
E           and: 'cachedir: .pytest_cache'
E           and: 'rootdir: /tmp/pytest-of-marcel/pytest-150/test_dist_tests_with_crash0'
E           and: 'plugins: xdist-3.4.0, relaxed-2.0.1'
E           and: 'collecting ... 1 worker [5 items]'
E           and: ''
E           and: 'scheduling tests via LoadScheduling'
E           and: 'fail0'
E           and: 'fail1'
E           and: 'ok'
E           and: 'skip'
E           and: '[gw0] node down: Not properly terminated'
E           and: ''
E           and: 'replacing crashed worker gw0'
E           and: ''
E           and: '=================================== FAILURES ==================================='
E           and: '__________________________________ test_fail0 __________________________________'
E           and: '[gw0] sunos5 -- Python 3.9.16 $(BUILD_DIR)/.tox/py39/bin/python'
E           and: ''
E           and: '    def test_fail0():'
E           and: '>       assert 0'
E           and: 'E       assert 0'
E           and: ''
E           and: 'test_dist_tests_with_crash.py:3: AssertionError'
E           and: '__________________________________ test_fail1 __________________________________'
E           and: '[gw0] sunos5 -- Python 3.9.16 $(BUILD_DIR)/.tox/py39/bin/python'
E           and: ''
E           and: '    def test_fail1():'
E           and: '>       raise ValueError()'
E           and: 'E       ValueError'
E           and: ''
E           and: 'test_dist_tests_with_crash.py:5: ValueError'
E           and: '________________________ test_dist_tests_with_crash.py _________________________'
E           and: '[gw0] sunos5 -- Python 3.9.16 $(BUILD_DIR)/.tox/py39/bin/python'
E           and: "worker 'gw0' crashed while running 'test_dist_tests_with_crash.py::test_crash'"
E           and: '=========================== short test summary info ============================'
E           and: 'FAILED test_dist_tests_with_crash.py::test_fail0 - assert 0'
E           and: 'FAILED test_dist_tests_with_crash.py::test_fail1 - ValueError'
E           and: 'FAILED test_dist_tests_with_crash.py::test_crash'
E           and: '==================== 3 failed, 1 passed, 1 skipped in 1.81s ===================='
E       remains unmatched: '*PASS**test_ok*'

$(BUILD_DIR)/testing/acceptance_test.py:216: Failed
----------------------------- Captured stdout call -----------------------------
created: 1/1 worker
============================= test session starts ==============================
platform sunos5 -- Python 3.9.16, pytest-7.4.3, pluggy-1.3.0 -- $(BUILD_DIR)/.tox/py39/bin/python
cachedir: .pytest_cache
rootdir: /tmp/pytest-of-marcel/pytest-150/test_dist_tests_with_crash0
plugins: xdist-3.4.0, relaxed-2.0.1
collecting ... 1 worker [5 items]

scheduling tests via LoadScheduling
fail0
fail1
ok
skip
[gw0] node down: Not properly terminated

replacing crashed worker gw0

=================================== FAILURES ===================================
__________________________________ test_fail0 __________________________________
[gw0] sunos5 -- Python 3.9.16 $(BUILD_DIR)/.tox/py39/bin/python

    def test_fail0():
>       assert 0
E       assert 0

test_dist_tests_with_crash.py:3: AssertionError
__________________________________ test_fail1 __________________________________
[gw0] sunos5 -- Python 3.9.16 $(BUILD_DIR)/.tox/py39/bin/python

    def test_fail1():
>       raise ValueError()
E       ValueError

test_dist_tests_with_crash.py:5: ValueError
________________________ test_dist_tests_with_crash.py _________________________
[gw0] sunos5 -- Python 3.9.16 $(BUILD_DIR)/.tox/py39/bin/python
worker 'gw0' crashed while running 'test_dist_tests_with_crash.py::test_crash'
=========================== short test summary info ============================
FAILED test_dist_tests_with_crash.py::test_fail0 - assert 0
FAILED test_dist_tests_with_crash.py::test_fail1 - ValueError
FAILED test_dist_tests_with_crash.py::test_crash
==================== 3 failed, 1 passed, 1 skipped in 1.81s ====================

Version info:

• pytest-relaxed 2.0.1
• pytest-xdist 3.4.0
• pytest 7.4.3
• pluggy 1.3.0
• Python 3.9.16

[mtelka]

Since pytest-relaxed is not needed to run pytest-xdist tests then we should not install (or load) it while testing pytest-xdist.