-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add stale issue workflow for flaky test reports #43547
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for putting this together - I've marked the PR as closing issue #37968.
There was some discussion before about not doing this because some flakey tests are skipped and still need to be fixed, so the issue will become stale and closed but the test will never be fixed.
But I think the volume is so much now that it's worth doing this anyway.
Thanks, @talldan. Somehow I missed that issue.
Maybe we can prevent this by using the |
Sounds like a good idea, it might be tricky to get contributors to understand the workflow, but worth trying. Maybe the stale or flaky messages can explain it. |
Also, considering that skipped tests require manual action, this might not be a big problem. I will merge workflow in the current state but keep an eye out for improvements. |
Fixes #37968
What?
PR adds a new workflow to check issues with the "[Type] Flaky Test" label and no activity in the past 30 days. Detected issues will be labeled "Stale" and closed in one day without activity.
Why?
I think it's safe to assume that the flaky tests report with no activity in the past 30 days might be resolved or was a false report due to some specific conditions.
Testing Instructions
The GitHub workflows are a bit hard to test, so verify settings based on the
actions/stale
docs - https://github.com/actions/stale/tree/v3.0.17#readme.