-
Notifications
You must be signed in to change notification settings - Fork 629
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use cfg_attr
to ignore expensive tests
#6032
Conversation
Definitely an improvement! I have doubts about introducing a proc macro for this, as this seems a too big of a solution for the problem at hand. I personally would strongly prefer open-coding
Admittedly, none of the arguments is particularly strong, so approving. |
One more weak contra argument -- this has to be a non-dev dependency, which feels not quite right. |
Rather than using `cfg` to not compile expensive tests, use `cfg_attr` to conditionally mark such tests as ignored. In other words, instead of writing: #[cfg(feature = "expensive_tests")] #[test] fn test_clear_old_data_too_many_heights() { // ... } write: #[test] #[cfg_attr(not(feature = "expensive_tests"), ignore)] fn test_clear_old_data_too_many_heights() { // ... } With this change, expensive tests will always be built which means that i) any code changes breaking them will be caught by CI (rather than having to wait for a nightly run) and ii) code used by expensive tests only is no longer unused when `expensive_tests` feature is not enabled (which means fewer `#[cfg(feature = "expensive_tests")]` directives sprinkled throughout code). Since we no longer mark whole modules as compiled only if the feature is enabled, this change also means that each individual test needs to be marked individually (rather than being able to mark whole module). This makes it more obvious which tests are expensive and which aren’t (since the marking is right at the test definition site) and simplifies `check_nightly.py` script. Issue: #4490
@matklad, PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I agree with @matklad this looks much better with proc macro. I found out that you can't run tests directly inside CLion, due to macros not being handled properly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
#[nightly_test]
attributecfg_attr
to ignore expensive tests
Rather than using
cfg
to not compile expensive tests, usecfg_attr
to conditionally mark such tests as ignored. In other words, instead
of writing:
write:
With this change, expensive tests will always be built which means
that i) any code changes breaking them will be caught by CI (rather
than having to wait for a nightly run) and ii) code used by expensive
tests only is no longer unused when
expensive_tests
feature is notenabled (which means fewer
#[cfg(feature = "expensive_tests")]
directives sprinkled throughout code).
Since we no longer mark whole modules as compiled only if the feature
is enabled, this change also means that each individual test needs to
be marked individually (rather than being able to mark whole module).
This makes it more obvious which tests are expensive and which aren’t
(since the marking is right at the test definition site) and
simplifies
check_nightly.py
script.Issue: #4490