CI: Mark s3 tests parallel safe #35895
Conversation
Closes pandas-dev#35856 I think we need to update the pytest pattern though, so this should fail.
Does this mean that this PR should not be merged despite being green? |
|
I'm going to rerun it a few times. I suspect we'll get a failure. |
TomAugspurger
changed the title
|
Updated to just make these fixtures parallel safe. At the start of each session, each worker starts up a moto server on I think Martin mentioned possible memory issues on windows workers with this approach, but that may have been starting the moto server at the test level. Hopefully doing things at the session level avoids that. |
|
Rerunning azure at https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=40904&view=results. But we should be good. |
TomAugspurger
added
CI
|
#35655 wasn't backported, so this shouldn't need a backport either. |
|
I've seen a couple failures on re-runs, but nothing to do with s3 / moto. So this should be good to go. |
| @@ -34,12 +34,13 @@ def feather_file(datapath): | |||
|
|
|||
|
|
|||
| @pytest.fixture | |||
| def s3so(): | |||
| return dict(client_kwargs={"endpoint_url": "http://127.0.0.1:5555/"}) | |||
| def s3so(worker_id): | |||
There was a problem hiding this comment.
@TomAugspurger where is this defined? I'm getting test failures locally, guessing i need to update some dependency
There was a problem hiding this comment.
worker_id is from pytest-xdist.
There was a problem hiding this comment.
test_s3_roundtrip_for_dir is failing in the CI and im trying to reproduce locally but it is getting pytest.skipped and it isn't clear why. could worker_id be involved?
Yes, this was for the case of restarting the process for every test (i.e., standard pytest scope) - I believe moto may be less than optimal at cleaning up on exit, at least on Windows. |
Closes #35856
I think we need to update the pytest pattern though, so this should
fail.