-
Notifications
You must be signed in to change notification settings - Fork 230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Load-Distribute test cases by filename #242
Conversation
The failing tests also fail on master. |
First of all thanks @Corni for the contribution!
I see. Perhaps the recent |
Hi, unfortunately, this does not help. We also do need several classes in the file, because we have a common body of tests we run in a shared superclass, which we then instantiate with different configurations, potentially overriding some tests to adjust for the different configuration. Yet, all of these use the shared redis instance. In the end, I would agree that this PR is a cheaper workaround around not having #18 implemented, a bit like loadscope is as well. |
Actually using a # test_redis.py
@pytest.fixture(autouse=True, scope='module')
def force_run_in_single_worker():
pass Now run with Alternatively, for your specific user case specifically, where you have a single
But I believe the solution using Sorry if it appears I'm rejecting your patch outright, I just want to make sure we understand the problem and perhaps propose an alternative solution first. |
Hi, I'm totally fine with this patch being rejected if the functionality is there, I understand the goal of keeping the codebase small and maintainable. Unfortunately, while I would be fine with the first solution (implicit module-level fixture), it doesn't work in my tests. import pytest
@pytest.fixture(autouse=True, scope="module")
def fix():
pass
class TestCaseA:
def test_one(self, fix):
assert True
class TestCaseB:
def test_two(self, fix):
assert True Which I start via: Which gives the following output, indicating that TestCaseA and TestCaseB have been scheduled to different executors, right?
The alternate solution is, unfortunately, a non-solution for my case, as nearly all other test files suffer from the same problem, just the shared ressource (redis in the example) is a MySQL-server, a Postgre-server etc. |
My bad, that's not how
class TestCaseA:
def test_one(self, fix):
assert True
def test_two(self, fix):
assert True
How are your tests organized inside the test files? Are they plain test classes or |
Having said that, the new addition is pretty trivial in terms of code and maintainability, we should consider adding it. @RonnyPfannschmidt what's your opinion on this? |
Hi, the test classes are plain test classes. |
97507e7
to
3060438
Compare
3060438
to
fad312f
Compare
ping - the failing tests also fail on master. |
Thanks @Corni for the ping! Unless @RonnyPfannschmidt needs more time to review this, I will look into merging this next week and make a release! 👍 |
@nicoddemus please go ahead |
@Corni I merged with the latest |
@Corni |
Hi,
this PR includes a new way of load-distributing tests by the file they reside in.
The implementation is a very simple change of LoadScopeScheduling.
The reason to have this is that in a project I'm working on, there is a single shared ressource per test file, which can't be used concurrently. Unfortunately, just scheduling the tests in one class on the same executor does not help, as there are different test classes, representing different configurations of the shared ressource in each file.
Thus, the proposed LoadFileScheduling.
changelog
folder following these guidelines:Name it
$issue_id.$type
for example588.bugfix
;If you don't have an issue_id change it to the PR id after creating it
Ensure type is one of
removal
,feature
,bugfix
,vendor
,doc
ortrivial
Make sure to use full sentences with correct case and punctuation, for example: