-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CN] Add benchmarking for PRs #537
Conversation
5a0b6ab
to
c2cf86f
Compare
c2cf86f
to
50481e2
Compare
@dc-mak The CI for benchmarking regressions fails due to a permission error when attempting to post a comment to the PR with the results. This is succeeding on our fork. I've added permissions to the workflow, but that does not seem to make a difference. Can you check if there are repository settings that are preventing comments from being posted? This post describes how to modify workflow permissions. |
@kmemarian can you check that post? The logs error-report also shows this link: https://docs.github.com/en/rest/issues/comments?apiVersion=2022-11-28#create-an-issue-comment |
@kmemarian, @dc-mak, @cp526 perhaps I can sit down with one of you and we can debug this together? |
@jprider63 That looks great! Happy to go over it together to get it to work, I'll email you. |
FYI I posted about the permission issues here. Hopefully we get a response! |
Thanks! |
This PR adds a CI workflow that adds regression benchmarks to PRs. It builds upon #530 so that should be merged first (ignore the first three commits).
Every time a PR is updated, a comment is posted to the PR that displays the old runtimes and the new runtimes. It highlights any regressions and if any benchmarks were removed. An example comment is here.
Implements part of rems-project/cn-tutorial#54.