Option to retry tests

[kpheng]

Tests should be consistent. They should either always fail or always pass. Unit tests are generally pretty consistent. With functional tests, however, several factors outside of our control may cause a test to fail. For example, my functional testing stack consists of MochaJS, WebDriverJS, and BrowserStack. The tests usually pass. However, in some cases, like when the network is congested or BrowserStack is slow in creating a VM session, a test may fail.

Since mocha doesn't support retries yet, I have to re-run test suites in their entirety. With each suite taking a non-trivial amount of time (between 5 to 15 minutes) to complete, this solution isn't ideal.

It would be nice if mocha supported retries. If a test in a suite fails, then I could retry that test instead of of re-running the entire suite.

[dasilvacontin]

👍 from me. Re-running all tests is a waste.

[keithamus]

Just an FYI @kpheng there are a couple of libraries out there that already to this. mocha-retry is one - which allows you to specify a number of retries per describe or it. Another is mocha-extra-shot which plugs directly into the runner which is a bit lower level.

[boneskull]

if this use case is covered by 3p projects, it doesn't need to be in mocha core

[jmnsf]

In case you end up here as I did: http://mochajs.org/#retry-tests

[boneskull]

3p = third party