Flaky fail on final test invokes pytest session fixture

[robertrmartinez]

I have a pytest fixture which runs at the "session" scope. This fixture starts a local web server, and initiates a sauce labs connection. Upon teardown, it stops the server and closes the sauce connection.

When the final test of my test run is a "flaky" test and fails, it acts as if the session is over and calls my fixture tear down.

Then, the flaky test is rerun, which invokes the session fixture once again.

I would like the session fixture to only be called once obviously due to the overhead of having to spin up the server and start the sauce connection again.

This could be a pytest issue, but I figured I would open an issue here as well to see if there is any suggestions or solutions in the works?

[AlexDubrovsky]

I have a similar issue with the following example:
`import pytest
from flaky import flaky

class TestFlaky():

@classmethod
def setup_class(cls):
    print("In setup class")

@flaky(max_runs=5, min_passes=1)
@pytest.mark.parametrize('param', [1, 2, 3])
def test1(self, param):
    print("In test1, param={}".format(param))
    assert param == 1

@classmethod
def teardown_class(cls):
    print("In teardown class")`

When the last test fails, the teardown is triggered, but then the whole sequence of setup->test->teardown is triggered again for the remaining attempts, only for the last test (with param=3).

[Sebelino]

Same issue here. A dumb workaround for now is to create a non-flaky dummy test at the end of the module/session scope:

@flaky
def my_last_real_test():
    ...

def test_nonflaky_dummy():  # Exists as a workaround for https://github.com/box/flaky/issues/128
    pass