Cypress Test spec timeout threshold

[mastrzyz]

What would you like?

I would like a configurable way to specify the maximum amount of time a test spec should take in seconds.

This should be on the spec level since we use batch execution of multiple specs in a suite to optimize for overhead.

eg with:
defaultSpecTimeout: 50

Spec 1: 40s -> No Exception

Spec 2: 54s -> Test Result failed and "Test has failed due to going over maximum duration threshold"

It is critical these tests still produce results in JUNIT and we continue execution.

Why is this needed?

We have problems with Uber tests being created that test multiple scenarios and edge cases, this is not a bad thing for coverage of the product but it makes test case sharding into multiple phases problematic. It also has a big problem that if the beginning of the test fails then the entire coverage is gone.

Creating a maximum spec threshold would create an upper bound for developers to follow when creating tests.

[bahmutov]

Would a plugin solution like https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/fundamentals__timeout work?

[mastrzyz]

@bahmutov let me check, did not know about it!

[mastrzyz]

@bahmutov that works on a test level but I need it on a spec level, ie a Cypress file.

We have large specs having many smaller tests which is causing us the problem.

[bahmutov]

@mastrzyz in that case you could use Cypress Module API to grab spec durations https://on.cypress.io/module-api and fail the run if any of the specs takes longer than necessary.

Alternatively, you could even have before and after hooks in the spec file to throw an error if the duration takes too long

[jennifer-shehane]

Closing as resolved.