Parallel test execution

[kvenkat88]

Is there any possibility to run test tests in parallel using tavern. Consider a scenario , through pytest parser utility functions receiving the input from command line and have to use it in our tests. For example, have command line flag as --user a b (input is in list). By using this test i have to cover the test scenarios for both users in parallel. Through pytest we can use pytest-xdist and parameterize option to do it. Refer the below links ,

https://stackoverflow.com/questions/49699477/is-it-possible-to-execute-the-same-test-in-parallel-in-pytest
https://stackoverflow.com/questions/47439103/how-to-execute-tests-in-a-test-suite-in-parallel
https://github.com/pytest-dev/pytest-xdist

Requirement would be like test scripts should be reusable and based upon the user type test scenarios has to be validated and reported in parallel. Right now fixture don't have support for parameterization support to do this like in Pytest.

[michaelboulton]

Tavern should work fine with pytest-xdist (I used to use it in the past, and just tried it now and it works fine with the integration tests), have you run into any problems with it?

eg:

plugins: tavern-0.22.1, xdist-1.26.0, remove-stale-bytecode-3.0, forked-1.0.1
[gw0] linux Python 3.6.7 cwd: /home/michael/code/tavern/tavern/tests/integration
[gw1] linux Python 3.6.7 cwd: /home/michael/code/tavern/tavern/tests/integration
[gw0] Python 3.6.7 (default, Oct 22 2018, 11:32:17)  -- [GCC 8.2.0]
[gw1] Python 3.6.7 (default, Oct 22 2018, 11:32:17)  -- [GCC 8.2.0]
gw0 [120] / gw1 [120]
scheduling tests via LoadScheduling

test_data_key.tavern.yaml::Test sending form encoded data works
test_data_key.tavern.yaml::Test sending raw data
[gw0] [  0%] PASSED test_data_key.tavern.yaml::Test sending form encoded data works
test_data_key.tavern.yaml::Test sending base64 data
[gw1] [  1%] PASSED test_data_key.tavern.yaml::Test sending raw data
test_data_key.tavern.yaml::Test sending a list in 'data' raises an error
[gw1] [  2%] PASSED test_data_key.tavern.yaml::Test sending a list in 'data' raises an error
test_dummy.py::test_nothing
[gw1] [  3%] PASSED test_dummy.py::test_nothing
test_files.tavern.yaml::Test files can be uploaded with tavern
[gw0] [  4%] PASSED test_data_key.tavern.yaml::Test sending base64 data
test_data_key.tavern.yaml::Test sending a float in 'data' raises an error
[gw1] [  5%] PASSED test_files.tavern.yaml::Test files can be uploaded with tavern
test_files.tavern.yaml::Test sending a text file will send the correct content type
[gw1] [  5%] PASSED test_files.tavern.yaml::Test sending a text file will send the correct content type
test_fixtures.tavern.yaml::Test usefixtures being a mapping errors
[gw0] [  6%] PASSED test_data_key.tavern.yaml::Test sending a float in 'data' raises an error
test_env_var_format.tavern.yaml::Test getting format vars from environment variables
[gw0] [  7%] PASSED test_env_var_format.tavern.yaml::Test getting format vars from environment variables
test_files.tavern.yaml::Test extra headers don't break content-type
[gw1] [  8%] PASSED test_fixtures.tavern.yaml::Test usefixtures being a mapping errors
test_fixtures.tavern.yaml::Test yielding fixture
[gw0] [  9%] PASSED test_files.tavern.yaml::Test extra headers don't break content-type
test_fixtures.tavern.yaml::Test empty usefixtures errors
[gw1] [ 10%] PASSED test_fixtures.tavern.yaml::Test yielding fixture
test_header_comparisons.tavern.yaml::Test mismatch in header value
[gw0] [ 10%] PASSED test_fixtures.tavern.yaml::Test empty usefixtures errors
test_fixtures.tavern.yaml::Test usefixtures
[gw0] [ 11%] PASSED test_fixtures.tavern.yaml::Test usefixtures                             
...

[kvenkat88]

Thanks for your reply great to hear from you. There is no documentation about parallel test execution through tavern. I can use all of the functionality available in pytest-xdist library .Could you please confirm.

[michaelboulton]

There isn't anything in Tavern that should break pytest-xdist, so you should be able to use it just fine. If you do run into any problems with it, feel free to make an issue about it.

[tommyjcarpenter]

@michaelboulton in pytest-xdist there is a marker to run certain tests in parallel, see pytest-dev/pytest-xdist#325
but that seems down at the pytest level, and tavern abstracts that level away (there is no pytest file, just the tavern)

if I want to run two tavern tests in parallel (gaurantee that my server gets two tests at the same time), how can I achieve this in the tavern yaml?

[tommyjcarpenter]

im gonna make my own issue