Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add workflow to determine API changes and comment on PRs #297

Merged

Conversation

Xtansia
Copy link
Collaborator

@Xtansia Xtansia commented May 17, 2024

Description

Adds a workflow that determines the API changes introduced by a PR and then comments on the PR with a summary of the changes a long with a link to a full HTML based report. If a comment already exists it will edit it rather than creating a new one.

Examples

Screenshot 2024-05-18 at 12 21 01 AM
Screenshot 2024-05-18 at 12 19 16 AM
Screenshot 2024-05-18 at 12 15 15 AM

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented May 17, 2024

API specs implemented for 307/649 (47%) APIs.
Commit c6f2cf2.

@dblock
Copy link
Member

dblock commented May 17, 2024

We already have https://github.com/opensearch-project/opensearch-api-specification/blob/main/.github/workflows/coverage-api.yml and https://github.com/opensearch-project/opensearch-api-specification/blob/main/.github/workflows/coverage-comment.yml, how do you feel about combining these? All you need is the interesting parts of .github/workflows/determine-changes.yml (can be in its own workflow) and the commenting logic of editing/updating is already fully implemented. Or move the coverage workflow gathering into this one?

One advantage of that workflow is that it will re-create the comment after the change, so that it's not confusing to which commit it refers.

@Xtansia
Copy link
Collaborator Author

Xtansia commented May 17, 2024

We already have https://github.com/opensearch-project/opensearch-api-specification/blob/main/.github/workflows/coverage-api.yml and https://github.com/opensearch-project/opensearch-api-specification/blob/main/.github/workflows/coverage-comment.yml, how do you feel about combining these? All you need is the interesting parts of .github/workflows/determine-changes.yml (can be in its own workflow) and the commenting logic of editing/updating is already fully implemented. Or move the coverage workflow gathering into this one?

One advantage of that workflow is that it will re-create the comment after the change, so that it's not confusing to which commit it refers.

When you say combine do you mean as a single comment containing both pieces of information or just combining logic/workflow?

I personally prefer editing the existing comment rather than deleting & recreating as it cuts down on the notification/email spam and also means you get "free" history record via the comment's edit history (albeit expiring after some time). This is how codecov does its coverage comments. As long as the comment body contains the sha then there's no confusion. It also means its likely to always be the first comment and stay there rather than moving over time and get lost between other comments.

@dblock
Copy link
Member

dblock commented May 17, 2024

When you say combine do you mean as a single comment containing both pieces of information or just combining logic/workflow?

I think we should combine both pieces of information. Changes and coverage are very related, I'd be happy if you want to take the coverage parts and add them to this PR (and delete the existing workflows).

I personally prefer editing the existing comment rather than deleting & recreating as it cuts down on the notification/email spam and also means you get "free" history record via the comment's edit history (albeit expiring after some time). This is how codecov does its coverage comments. As long as the comment body contains the sha then there's no confusion. It also means its likely to always be the first comment and stay there rather than moving over time and get lost between other comments.

Sounds good, no strong feelings about this.

@Xtansia Xtansia marked this pull request as draft May 20, 2024 02:37
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good.

Write up how these things work in various READMEs (README, dev guide, etc.), especially cookbooks on running some of the coverage itself locally so that one doesn't have to dig through workflow files for examples to copy-paste from.

fi
echo '=> waiting...'
done
exit 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're writing tools here, would add a JS tool to wait for OpenSearch to start that polls.

name: coverage-api
path: |
/tmp/coverage-api-ORIGINAL.json
/tmp/coverage-api-CHANGED.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use SHAs instead of ORIGINAL and CHANGED to avoid confusion, running over old data, etc.

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
@Xtansia Xtansia force-pushed the feat/determine-api-changes-workflow branch from e40e32b to 8a10252 Compare May 21, 2024 02:00
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
@Xtansia Xtansia force-pushed the feat/determine-api-changes-workflow branch from fb87d2f to 5b2db1d Compare May 21, 2024 05:36
@Xtansia
Copy link
Collaborator Author

Xtansia commented May 21, 2024

@dblock I still need to do the documentation, however I've rearchitected things a bit further and now have comments like below. I think this would be a good point to get some of your feedback on how you feel about the approach before I write everything up now that it's basically complete. I've potentially gone overkill with trying to generalize the actual PR commenting side of the workflow. 😅

Screenshot 2024-05-21 at 5 34 57 PM

dblock
dblock previously approved these changes May 21, 2024
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty crazy but I like it. Avoid more feature creep and finish this PR :)

@@ -0,0 +1,23 @@
# Changes Analysis
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd name this templates for future templates.

@dblock
Copy link
Member

dblock commented May 21, 2024

In general, I think this is a good blueprint for a service that provides OpenAPI spec coverage, sounds like a nice pet project!

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
@Xtansia Xtansia force-pushed the feat/determine-api-changes-workflow branch from 98e6190 to ac18261 Compare May 22, 2024 05:33
@Xtansia Xtansia marked this pull request as ready for review May 22, 2024 05:33
@Xtansia Xtansia requested a review from dblock May 22, 2024 05:33
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
@dblock dblock merged commit c58908d into opensearch-project:main May 22, 2024
6 checks passed
@Xtansia Xtansia deleted the feat/determine-api-changes-workflow branch May 22, 2024 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants