Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-97923: Always run Ubuntu SSL tests with others in CI #97940

Merged
merged 1 commit into from
Oct 7, 2022

Conversation

sobolevn
Copy link
Member

@sobolevn sobolevn commented Oct 5, 2022

Now Ubuntu SSL tests will be run whenever other tests are run.
But, it will still ignore doc only changes (as it is now).

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, but I don't know what these rules do. Hopefully there's another reviewer you can add (IIRC @tiran is offline this week).

@vstinner
Copy link
Member

vstinner commented Oct 6, 2022

Please try to fix the issue below on the OpenSSL jobs before merging this PR:

1 test altered the execution environment:
    test_asyncio

@sobolevn
Copy link
Member Author

sobolevn commented Oct 6, 2022

Please try to fix the issue below on the OpenSSL jobs before merging this PR:

Sorry, but I don't know what is going on there, as far as I understand it is a work-in-progress by @arhadthedev

@arhadthedev
Copy link
Member

Thank you, I'll dig into it in two hours.

To find the reason, I need to teach the data supplier of this code:

if self.environment_changed:
print()
print("{} altered the execution environment:".format(
count(len(self.environment_changed), "test")))
printlist(self.environment_changed)

to trigger after individual test cases, not between suit files.

@arhadthedev
Copy link
Member

@vstinner Could you link to the log please? It looks like a misunderstanding because I looked into the checks for this PR and there're other results:

  • https://github.com/python/cpython/actions/runs/3193195105/jobs/5211517211#step:6:921 (Windows (x64)):

    2 tests altered the execution environment:
        test_distutils test_peg_generator
    
    [...]
    
    Tests result: FAILURE then SUCCESS
    
  • https://github.com/python/cpython/actions/runs/3193195105/jobs/5211516860#step:17:674 (Ubuntu):

    == Tests result: SUCCESS ==
    
    426 tests OK.
    
    10 slowest tests:
    - test_tools: 6 min
    - test_multiprocessing_spawn: 4 min 41 sec
    - test_concurrent_futures: 2 min 49 sec
    - test_multiprocessing_fork: 2 min 7 sec
    - test_asyncio: 1 min 48 sec
    - test_gdb: 1 min 34 sec
    - test_multiprocessing_forkserver: 1 min 33 sec
    - test_venv: 1 min 23 sec
    - test_signal: 1 min 3 sec
    - test_buffer: 52.4 sec
    
    11 tests skipped:
        test_devpoll test_ioctl test_kqueue test_launcher test_msilib
        test_startfile test_winconsoleio test_winreg test_winsound
        test_wmi test_zipfile64
    
    Total duration: 11 min 57 sec
    Tests result: SUCCESS
    

I also tried to reproduce it locally on Windows 10 21H2 using main (6d0a019). An output of pcbuild\clean && pcbuild\build && python -m test -v test_asyncio:

[...]

Ran 2010 tests in 111.019s

OK (skipped=64)
test_asyncio passed in 1 min 54 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1 min 56 sec
Tests result: SUCCESS

gvanrossum pushed a commit that referenced this pull request Oct 7, 2022
…_streams: port server cases to IsolatedAsyncioTestCase" (#98015)

This PR reverts gh-93369 and gh-97896 because they've made asyncio tests unstable. After these PRs were merged, random GitHub action jobs of random commits started to fail unrelated tests and test framework methods.

The reverting is necessary because such shrapnel failures are a symptom of some underlying bug that must be found and fixed first.

I had a hope that it's a server overload because we already have extremely rare disc access errors. However, one and a half day passed, and the failures continue to emerge both in PRs and commits.

Affected issue: gh-93357.
First reported in #97940 (comment).

* Revert "gh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (#97896)"

This reverts commit 09aea94.

* Revert "gh-93357: Start porting asyncio server test cases to IsolatedAsyncioTestCase (#93369)"

This reverts commit ce8fc18.
@ambv ambv merged commit 3108fc1 into python:main Oct 7, 2022
carljm added a commit to carljm/cpython that referenced this pull request Oct 8, 2022
* main: (38 commits)
  pythongh-92886: make test_ast pass with -O (assertions off) (pythonGH-98058)
  pythongh-92886: make test_coroutines pass with -O (assertions off) (pythonGH-98060)
  pythongh-57179: Add note on symlinks for os.walk (python#94799)
  pythongh-94808: Fix regex on exotic platforms (python#98036)
  pythongh-90085: Remove vestigial -t and -c timeit options (python#94941)
  pythonGH-83901: Improve Signature.bind error message for missing keyword-only params (python#95347)
  pythongh-61105: Add default param, note on using cookiejar subclass (python#95427)
  pythongh-96288: Add a sentence to `os.mkdir`'s docstring. (python#96271)
  pythongh-96073: fix backticks in NEWS entry (pythonGH-98056)
  pythongh-92886: [clinic.py] raise exception on invalid input instead of assertion (pythonGH-98051)
  pythongh-97997: Add col_offset field to tokenizer and use that for AST nodes (python#98000)
  pythonGH-88968: Reject socket that is already used as a transport (python#98010)
  pythongh-96346: Use double caching for re._compile() (python#96347)
  pythongh-91708: Revert params note in urllib.parse.urlparse table (python#96699)
  pythongh-96265: Fix some formatting in faq/design.rst (python#96924)
  pythongh-73196: Add namespace/scope clarification for inheritance section (python#92840)
  pythongh-97646: Change `.js` and `.mjs` files mimetype to conform to RFC 9239 (python#97934)
  pythongh-97923: Always run Ubuntu SSL tests with others in CI (python#97940)
  pythongh-97956: Mention `generate_global_objects.py` in `AC How-To` (python#97957)
  pythongh-96959: Update HTTP links which are redirected to HTTPS (python#98039)
  ...
mpage pushed a commit to mpage/cpython that referenced this pull request Oct 11, 2022
…o.test_streams: port server cases to IsolatedAsyncioTestCase" (python#98015)

This PR reverts pythongh-93369 and pythongh-97896 because they've made asyncio tests unstable. After these PRs were merged, random GitHub action jobs of random commits started to fail unrelated tests and test framework methods.

The reverting is necessary because such shrapnel failures are a symptom of some underlying bug that must be found and fixed first.

I had a hope that it's a server overload because we already have extremely rare disc access errors. However, one and a half day passed, and the failures continue to emerge both in PRs and commits.

Affected issue: pythongh-93357.
First reported in python#97940 (comment).

* Revert "pythongh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (python#97896)"

This reverts commit 09aea94.

* Revert "pythongh-93357: Start porting asyncio server test cases to IsolatedAsyncioTestCase (python#93369)"

This reverts commit ce8fc18.
mpage pushed a commit to mpage/cpython that referenced this pull request Oct 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants