Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report results of running pandas unit tests in CI and fail the job when necessary #14404

Open
shwina opened this issue Nov 14, 2023 · 3 comments
Labels
cudf.pandas Issues specific to cudf.pandas feature request New feature or request

Comments

@shwina
Copy link
Contributor

shwina commented Nov 14, 2023

Every PR runs a job called pandas-tests that run the Pandas unit tests using cudf.pandas. Currently, there are two issues with this job:

  • the results of those tests aren't reported in the job summary or anywhere convenient to look at
  • the job always passes, regardless of how many tests were passed or failed

Ideally, we should post the results as a job summary and fail the job if the number of tests passed relative to the development branch falls below a threshold (say, 0.1%).

@shwina shwina added feature request New feature or request pandas labels Nov 14, 2023
@galipremsagar galipremsagar added the cudf.pandas Issues specific to cudf.pandas label Apr 15, 2024
@vyasr
Copy link
Contributor

vyasr commented May 15, 2024

The reporting piece of this was addressed in #15369 and follow-ups. @galipremsagar do we want to do the other part and add a failure condition as well?

@galipremsagar
Copy link
Contributor

We currently use nightly reports as baseline, so we can't have a failure condition based on a threshold as it can swing widely based on the number of PRs being merged on a given day. We can revisit the second part when we have a more accurate(upto-date) branch based baselines.

@vyasr
Copy link
Contributor

vyasr commented May 15, 2024

Agreed, I don't think we can do this quite yet.

@vyasr vyasr removed the pandas label May 16, 2024
@vyasr vyasr removed the ci label Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cudf.pandas Issues specific to cudf.pandas feature request New feature or request
Projects
Status: In Progress
Development

No branches or pull requests

3 participants