Skip test case in pypy or windows

[ocelotl]

Fixes #3985

[xrmx]

I would bump the delta to 30 instead

[ocelotl]

I would bump the delta to 30 instead

Next time it can fail with a delta of 31 🤷 I don't see much sense in chasing this issue by increasing the delta, as far as we know this can fail with pretty much any delta but we are more certain that this happens only on Pypy or Windows.

[xrmx]

I would bump the delta to 30 instead

Next time it can fail with a delta of 31 🤷 I don't see much sense in chasing this issue by increasing the delta, as far as we know this can fail with pretty much any delta but we are more certain that this happens only on Pypy or Windows.

I think it would be nice to try to keep coverage on Windows I guess

[ocelotl]

I would bump the delta to 30 instead

Next time it can fail with a delta of 31 🤷 I don't see much sense in chasing this issue by increasing the delta, as far as we know this can fail with pretty much any delta but we are more certain that this happens only on Pypy or Windows.

I think it would be nice to try to keep coverage on Windows I guess

It would be great to have this test run on Windows, but it randomly fails there and that is out of our control. By the way, here here it failed with a delta of 34:

self.assertAlmostEqual((export_time - start_time) * 1e3, 500, delta=25)
E       AssertionError: 534.8496437072754 != 500 within 25 delta (34.84964370727539 difference)

So, either we skip this test case, or we automate what we always do, which is rerunning this test case everytime it fails.

[ocelotl]

I modified this PR so that the test case will be rerun if it fails (3 times maximum) with an AssertionError and if being run in Windows or with PyPy.

[ocelotl]

Ok, I am back to my original plan of just skipping this test case for pypy or windows. I also don't like to lower test coverage but there is no solution.

I tried to rerun this test case, it failed 3 times in a row: https://github.com/open-telemetry/opentelemetry-python/actions/runs/9782117611/job/27007817604?pr=3986#step:7:739

Also, trying to bump the delta is a naive approach: https://github.com/open-telemetry/opentelemetry-python/actions/runs/9782117611/job/27007817604?pr=3986#step:7:741
It failed with a delta of 2755! 🤦 3255.549192428589 != 500 within 40 delta (2755.549192428589 difference)

[ocelotl]

I tried rerunning the test cases with a 1s delay between runs, still failed. I got this idea from pytest-retry which includes a delay option.

[ocelotl]

@xrmx please take a look, I tried rerunning the test case, it still fails. If we find a solution to make this test case run consistently in Windows, then we can add that solution in a subsequent PR.

[ocelotl]

This test case failed in windows and py39 with a delta > 700 here.