Analyze and improve test_asyncio.py: right now it might be flaky

[sobolevn]

Openning as proposed in #97535 (comment)

We had multiple issues with test_asyncio before. Some tests are changing env, some are just flaky:

• asyncio pgo + lto buildbot failure: altered environment #97535
• Running tests in parallel on Windows quits too soon #95027
• test_asyncio environment changed during test execution due to empty threading._dangling without MainThread before some tests #91676
• [Windows] : Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure #85848
• flaky test.test_asyncio.test_events.ProactorEventLoopTests.test_call_later #85801
• test_asyncio: test_start_tls_server_1() fails randomly #76639
• Any others?

It is proposed to analyze what is causing this to happen again and again.
And fix it, of course :)

Linked PRs

• GH-97546: fix flaky asyncio test_wait_for_race_condition test #102421

[sobolevn]

This might be related: #93357

[gvanrossum]

Another possible source of flakiness is the use of small timeouts in tests, like 0.1 or even 0.01 seconds. Making the timeouts larger makes the tests run slower, but the short timeouts occasionally flake out. There should be other ways to wait for certain things to happen in most cases, like creating an Event object and awaiting that, instead of sleeping.

Longer timeouts (e.g. 1, 2 or 3 seconds) are less likely to be suspicious, timeouts like that are meant to be never hit unless the test is actually failing. Making these too long would just cause tests to take longer to report failure.

[kumaraditya303]

See also #82504

[arhadthedev]

test_asyncio is flaky again on ARM64 macOS randomly failing test_wait_for_race_condition. From https://buildbot.python.org/all/#builders/725/builds/3837 and https://buildbot.python.org/all/#builders/725/builds/3843:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/asyncio/tasks.py", line 478, in wait_for
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/unittest/async_case.py", line 90, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/unittest/async_case.py", line 112, in _callMaybeAsync
    return self._asyncioRunner.run(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_asyncio/test_waitfor.py", line 163, in test_wait_for_race_condition
    res = await task
          ^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/asyncio/tasks.py", line 477, in wait_for
    async with timeouts.timeout(timeout):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/asyncio/timeouts.py", line 98, in __aexit__
    raise TimeoutError
TimeoutError