Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate jest timeouts to @fast-check/jest #3084

Closed
CMCDragonkai opened this issue Aug 19, 2022 · 4 comments · Fixed by #3511
Closed

Integrate jest timeouts to @fast-check/jest #3084

CMCDragonkai opened this issue Aug 19, 2022 · 4 comments · Fixed by #3511

Comments

@CMCDragonkai
Copy link

🚀 Feature Request

Jest tests themselves have a timeout. It would be nice if these timeouts that are added into the testProp was used for fast check's timeout.

Motivation

This makes it easier to know that the fast check interruptAfterTimeLimit will be always be under the timeout given to the full jest test itself. I suggest that it may need to be the jest timeout minus 1 second.

Example

import { itProp, testProp, fc } from '@fast-check/jest';

describe('', () => {
  itProp('should run too', [fc.nat(), fc.nat()], (a, b) => {
    return a + b === b + a;
  }, 5000);
});

The 5000 should then be used for the interruptAfterTimeLimit.

@dubzzz
Copy link
Owner

dubzzz commented Sep 1, 2022

Definitely something to that could be interesting. fast-check's already receives a time-out as part of its options, it could either leverage it to set the one of jest or the opposite.

@CMCDragonkai
Copy link
Author

If I'm using the jest package, how does the timeout and interruptAfterTimeLimit work with jest's global default timeout.

Like if jest.setTimeout(20000);, then if I use timeout that should override it just like if I used timeout parameter with the test function.

Then if I use interruptAfterTimeLimit, does that also override the jest.setTimeout()?

@dubzzz
Copy link
Owner

dubzzz commented Sep 23, 2022

So far: timeout and interruptAfterTimeLimit of fast-check are fully disconnected from jest.setTimeout(). What I usually do for projects using both is:

// in jest.setup.cjs
fc.configureGlobal({timeout: 4000});
// alternatively: fc.configureGlobal({timeout: 4000, interruptAfterTimeLimit:4000, markInterruptAsFailure: false});
jest.setTimeout(5000)

See https://github.com/dubzzz/fast-check/blob/296cde419382eac6fe257ac4105a893dbdf5cfcc/packages/fast-check/documentation/Tips.md#setup-global-settings

@dubzzz
Copy link
Owner

dubzzz commented Dec 18, 2022

Probably one of the next feature I'm gonna to tackle

@dubzzz dubzzz pinned this issue Dec 18, 2022
dubzzz added a commit that referenced this issue Dec 22, 2022
In the context of #3084, we want to have a way built-in in fast-check to make sure that tests do not run for too long and can abide by certain timeouts constraints requested by the users.

While today we do have a timeout, it only scopes the interruption to one run of the predicate but does not consider "the runs" as a whole contrary to `interruptAfterTimeLimit`.

On the other hand, `interruptAfterTimeLimit` does have a timeout like behaviour but once started the predicate is never interrupted. Now it could be interrupted in-between, if running it until the end would cause a timeout given the time constraint passed to `interruptAfterTimeLimit`.

With such new feature in place, we will probably have to consider properties not having run at least once as failure, otherwise a buggy async algorithm would be marked as failure if it led to timeout at first run with `interruptAfterTimeLimit`.
dubzzz added a commit that referenced this issue Dec 22, 2022
* ✨ Interrupt predicates when `interruptAfterTimeLimit`

In the context of #3084, we want to have a way built-in in fast-check to make sure that tests do not run for too long and can abide by certain timeouts constraints requested by the users.

While today we do have a timeout, it only scopes the interruption to one run of the predicate but does not consider "the runs" as a whole contrary to `interruptAfterTimeLimit`.

On the other hand, `interruptAfterTimeLimit` does have a timeout like behaviour but once started the predicate is never interrupted. Now it could be interrupted in-between, if running it until the end would cause a timeout given the time constraint passed to `interruptAfterTimeLimit`.

With such new feature in place, we will probably have to consider properties not having run at least once as failure, otherwise a buggy async algorithm would be marked as failure if it led to timeout at first run with `interruptAfterTimeLimit`.

* versions

* Add tests

* add tests

* Revert "Add tests"

This reverts commit 14fb071.
@dubzzz dubzzz unpinned this issue Dec 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants