Adds feature where if review requirement not met, then request #30653

gfog-floqast · 2023-05-11T18:45:50Z

Fixes #

Proposed changes:

Adds feature where if a review requirement is not met, then request the review. This is beneficial so that a developer does not need to look at the review requirement(s) to determine who to request reviews from in the case that review requirements are not met.

Other information:

Have you written new tests for your changes, if applicable?
Have you checked the E2E test CI results, and verified that your changes do not break them?
Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

Create .github/required-review.yaml similar to the following:

- name: Terraform
  paths:
    - '**/*.tf'
  teams:
    - any-of:
      - all-of:
        - team1
        - team2
      - '@user1'

- name: CODEOWNERS
  paths:
    - 'CODEOWNERS'
  teams:
    - team1
    - '@user2'

- name: .github
  paths:
    - '.github/**'
  teams:
    - all-of:
      - team1
      - team2

Make updates to each of the files, as needed
Observe that on first pass the necessary teams/users are requested

Tested this by deploying fork to my local github env and using settings above.

Observed initial failure on all requirements and observed required reviews were requested:
Received review from team1 and observed CODEOWNERS check passed:
Received review from user1 and observed Terraform check passed:
Received review from team2 and observed .github check passed:
Additionally received review from user2, which had no affect on status.

github-actions · 2023-05-11T18:49:07Z

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

✅ Include a description of your PR changes.
✅ All commits were linted before commit.
✅ Add testing instructions.
✅ Specify whether this PR includes any changes to data or privacy.
✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

jeherve

First of all, thank you for the contribution!

I like that addition, I can see how useful it can be. However, since it can generate additional noise for maintainers since it will generate notifications and / or emails when they get added as reviewers, I think this option should be opt-in. I would consequently recommend that this only be available when specifying it in the action configuration.

What do you think of that?

jeherve · 2023-05-12T15:26:14Z

projects/github-actions/required-review/CHANGELOG.md

+## [3.1.0] - 2023-05-11
+### Added
+- Added feature where if a review requirement is not met, then request the review.
+
+### Changed
+- Updates misspelling


Good news, you do not need to fill this in. This changelog will be automatically updated when we next release a new version of this action, based off the different changelog entries in the changelog/ directory (where you've added your own changelog entry already, thank you).

Suggested change

## [3.1.0] - 2023-05-11

### Added

- Added feature where if a review requirement is not met, then request the review.

### Changed

- Updates misspelling

jeherve · 2023-05-12T15:36:17Z

projects/github-actions/required-review/src/request-review.js

+        return res.data.login
+    } catch ( error ) {
+        throw new WError(
+            // prettier-ignore


I see you've ignored Prettier here and across the file in general. Could you remove the exception and run it, so the style remains consistent with the other files in the action?

jeherve · 2023-05-12T15:36:55Z

projects/github-actions/required-review/src/request-review.js

+		// Handle @singleuser virtual teams. Fetch the correct username case from GitHub
+		// to avoid having to worry about edge cases and Unicode versions and such.
+        try {
+            const login = await getUsername(team);


I like that you extracted that here, that's clean. 👍

I wonder if we should make further use of that extraction in the existing logic in projects/github-actions/required-review/src/team-members.js?

yup, I can pull this out. Is there a file or location that is preferred for this? ie a helper file, or I can call it get-username.js and reference it in both files.

or I can call it get-username.js and reference it in both files

This seems like a good approach to me 👍

gfog-floqast · 2023-05-12T17:46:22Z

Thanks @jeherve, I'll look into making this feature opt-in

jeherve

I've pushed a commit to address some linting errors, hopefully that helps a bit.

I think we're still missing a check for the new request-reviews action parameter you added, before we try to request reviews.

gfog-floqast · 2023-05-15T16:24:10Z

@jeherve Thanks for addressing those linting errors. Regarding the // prettier-ignore comment, is there anything you would like me to do with it? I copied that from the team-members file so not sure if this needs an update.

jeherve · 2023-05-15T16:28:57Z

Regarding the // prettier-ignore comment, is there anything you would like me to do with it? I copied that from the team-members file so not sure if this needs an update.

I'll let @anomiex weigh in on that, since he worked on this part of the code in #29322

gfog-floqast · 2023-05-15T16:41:15Z

missing a check for the new request-reviews action parameter

Can I get more clarification on this, I'm not sure where I should be looking. thx

jeherve · 2023-05-15T16:46:30Z

Can I get more clarification on this, I'm not sure where I should be looking. thx

Sure thing. Here, you add a review request when needed:
https://github.com/Automattic/jetpack/pull/30653/files#diff-d99dd1c9d319cf9914b2111b19b7bb88771bcb4e4cee56ace33a4a95812bda3eR36

Here, you added a new Action parameter, request-reviews, so someone setting up the GitHub action in their own repo can decide to request reviews or not:
https://github.com/Automattic/jetpack/pull/30653/files#diff-b6d548c4bd130b59b0d693c1bd892f31aec433b2abfdd7067dc4f3f9ca5b031bR26

But you do not check for that parameter later on, so that parameter is useless right now. I would recommend that you check if that parameter is set before to call requestReview here:
https://github.com/Automattic/jetpack/pull/30653/files#diff-d99dd1c9d319cf9914b2111b19b7bb88771bcb4e4cee56ace33a4a95812bda3eR36

gfog-floqast · 2023-05-15T16:54:05Z

garr - missed it in my commit :D - It's there now

anomiex

Regarding the // prettier-ignore comment, is there anything you would like me to do with it? I copied that from the team-members file so not sure if this needs an update.

I'll let @anomiex weigh in on that, since he worked on this part of the code in #29322

I don't like when prettier decides to wrap a backquoted string over multiple lines, in large part because vim doesn't handle it very well. So I tend to prettier-ignore when that happens.

anomiex · 2023-05-15T17:41:25Z

projects/github-actions/required-review/src/requirement.js

+			if ( reviewSatisfied.length === 0 && request == 'true' ) {
+				await requestReview( team );
+			}
+			return printSet( `${ indent }Members of ${ team }:`, reviewSatisfied );


This is not the right place to add the requesting. If the configuration is requiring a review from "teamA or teamB", and someone from teamB already reviewed, there's no need to request a review from teamA. Also putting it here may result in redundant API calls if teamA is listed in multiple requirement blocks, and will almost certainly result in more API calls than are needed.

IMO the best thing to do would be, somewhere in

jetpack/projects/github-actions/required-review/src/main.js

Lines 84 to 88 in 8602a93

} else {

ok = false;

core.endGroup();

core.error( `Requirement "${ r.name }" is not satisfied by the existing reviews.` );

}

add the list of needed teams for r to a Set. You'll need to figure out how to get just the needed teams, of course. For example, if review from both teamX and ( teamY or userBob ) is required, and there's already a review from teamY, you'd ideally only want to request teamX, not teamY again and not userBob either.

Then, just after

jetpack/projects/github-actions/required-review/src/main.js

Lines 93 to 96 in 8602a93

await reporter.status(

core.getBooleanInput( 'fail' ) ? reporter.STATE_FAILURE : reporter.STATE_PENDING,

reviewers.length ? 'Awaiting more reviews...' : 'Awaiting reviews...'

);

if the new input is true and the set isn't empty, print a message along the lines of Requesting reviews from ${ members of the set } then pass the whole set to requestReviewer. Since the request reviewers API endpoint accepts multiple reviewers in one call, only one call should need to be made.

Thanks for the review @anomiex After going through the code I went this route due to how reviewers seemed to be grabbed first and then it looked like it worked back to the teams. This route seemed the most direct w/o making major changes to the rest of the code. With that said, I agree 💯 with limiting unneeded API calls and will look into reducing these as you're suggesting.

I'm wondering if it might be reasonable to say that review requests only occur on the first run? On that initial run I think it's safe to say that there would be no reviews therefor all required reviewers should be requested. Then as reviews are received, there shouldn't be the further need to request them again, as you point out. There could be a case where a review was removed (new commit push, etc), but thinking of how codeowners files works, it only requests reviews the first time iirc, which is the functionality I'm looking to introduce. Not sure if this would make the above any more straight forward.

If someone pushes a commit to a PR that touches additional files, that might also trigger a need for more reviewers.

For example, if someone from teamX says "oh, you also need to change this other thing over here" in their review, and that other thing happens to require review from teamZ.

yup - great point. 👍🏻

anomiex · 2023-05-15T17:42:50Z

projects/github-actions/required-review/src/get-username.js

+ *
+ * @param {string} user - @ followed by a GitHub user name.
+ */
+async function getUsername( user ) {


Add caching in here please. If called twice for the same user, it shouldn't hit the API again.

anomiex · 2023-05-15T18:00:28Z

projects/github-actions/required-review/src/request-review.js

+		// Handle @singleuser virtual teams. Fetch the correct username case from GitHub
+		// to avoid having to worry about edge cases and Unicode versions and such.


Is that a concern here? We do it in the other place because we're comparing the name from the config with the name returned on the review. But here I'd hope submitting "someusername" instead of "someUserName" would just do the right thing.

Tested, and yes someUserName works just as well as someusername. So guess there's really no reason to factor out this code as here we can just use const login = team.slice( 1 ); and not touch team-members.js.

If you agree, I'll just put that code back so I'm touching fewer files.

Sounds good to me!

gfog-floqast · 2023-05-23T16:43:42Z

Please let me know what you think of this approach. The main issue that I'm running into is accounting for 1) a nested all-of inside an any-of along with 2) a nested any-of inside an all-of. I tried creating a way to only add the alls in 1) if the any was not satisfied, but this then caused issues in 2) where the all would not get added if the any was satisfied. Without some sort of messy counter I'm not sure how best to handle this situation.

Currently I'm thinking the most optimal solution would be to add the all-of to the list for review request and if it gets an unneeded request that would be better than having a needed request not getting added. I can also add a check to see if a review is already requested and then not request again. This would lower notifications, but may cause issues when a new file needed to be touched that included a reviewer from scenario 1) who would now be required again.

I'm open to other suggestions of course, I started running in circles so thought best to reach out. This is my first time working with javascript so I may not be familiar with some other tricks that exist in this language for handing a situation like this.

gfog-floqast · 2023-05-23T16:46:36Z

projects/github-actions/required-review/src/requirement.js

@@ -166,7 +178,7 @@ class Requirement {
 			);
 		}

-		this.reviewerFilter = buildReviewerFilter( config, { 'any-of': config.teams }, '  ' );
+		this.reviewerFilter = buildReviewerFilter( config, 'any-of', { 'any-of': config.teams }, '  ' );


This is suggestion, if y'all agree with this solution I would look to update { 'any-of': config.teams } to just be config.teams and rework the related code.

gfog-floqast · 2023-05-23T16:48:42Z

Please let me know what you think of this approach. The main issue that I'm running into is accounting for 1) a nested all-of inside an any-of along with 2) a nested any-of inside an all-of. I tried creating a way to only add the alls in 1) if the any was not satisfied, but this then caused issues in 2) where the all would not get added if the any was satisfied. Without some sort of messy counter I'm not sure how best to handle this situation.

Currently I'm thinking the most optimal solution would be to add the all-of to the list for review request and if it gets an unneeded request that would be better than having a needed request not getting added. I can also add a check to see if a review is already requested and then not request again. This would lower notifications, but may cause issues when a new file needed to be touched that included a reviewer from scenario 1) who would now be required again.

I'm open to other suggestions of course, I started running in circles so thought best to reach out. This is my first time working with javascript so I may not be familiar with some other tricks that exist in this language for handing a situation like this.

anomiex · 2023-05-23T17:54:27Z

I don't like the globals inside requirements.js. I think that trying to keep them straight is probably where you're getting confused.

The functions returned from buildReviewerFilter() currently return an array of found reviewers, and when things get to the top if the result is non-empty then the requirement is satisfied. Let's change that so the functions now return two values (as an object with two keys): the array of found reviewers and an array or set of needed teams.

The simple string case will either return a non-empty array of found reviewers and an empty array/set of needed teams, or an empty array of found reviewers and a 1-element array/set containing the passed-in team.
The "any-of" and "all-of" cases would both calculate their arrays of found reviewers as they do now. Then if the found array is empty they will merge the needed-teams returned by all the sub-filters to return, otherwise they'll return an empty array/set for needed-teams.

Then I'd replace isSatisfied() with a needsReviewsFrom() method that returns the needed-teams set instead of a boolean. The code in main.js would then look something like this

} else {
	const neededForRequirement = await r.needsReviewsFrom( reviewers );
	core.endGroup();
	if ( neededForRequirement.size === 0 ) {
		core.info( `Requirement "${ r.name }" is satisfied by the existing reviews.` );
	} else {
		core.error( `Requirement "${ r.name }" is not satisfied by the existing reviews.` );
		// Merge into `neededTeams` set for possible use after the loop.
		neededForRequirement.forEach( neededTeams.add, neededTeams );
	}
}

Instead of ok, before the loop we'd initialize const neededTeams = new Set() and the post-loop check would look for neededTeams.size === 0.

gfog-floqast · 2023-06-01T15:50:14Z

@jeherve @anomiex I've made updates to address the comments, please let me know if there are any further changes or additions you would like to see. Thanks.

anomiex

Looks pretty good. I left a few comments inline, and I'll kick off the CI jobs where the linter will have some additional comments.

projects/github-actions/required-review/changelog/request-review-if-not-satisfied

projects/github-actions/required-review/src/request-review.js

projects/github-actions/required-review/src/requirement.js

Comments addressed.

…ew-if-not-satisfied Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

gfog-floqast · 2023-06-02T20:48:38Z

@anomiex Addressed comments and committed suggestions - I didn't get the lint results before committing suggested changes so I lost those runs if you could please retrigger those I'll make any needed updates.

gfog-floqast · 2023-06-07T16:10:38Z

@jeherve @anomiex could one of y'all re-trigger the workflows for me? please and thank you.

gfog-floqast · 2023-06-07T19:16:26Z

@jeherve Linting errors should be fixed and ready for a final 🤞🏻 run.

projects/github-actions/required-review/src/request-review.js

Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

gfog-floqast · 2023-06-09T17:01:28Z

Thanks @anomiex I committed your suggestion, also pushed up final linting error corrections.

anomiex

If CI is happy, let's go for it. Thanks!

github-actions bot added [Action] Required Review Actions GitHub actions used to automate some of the work around releases and repository management Docs labels May 11, 2023

github-actions bot added [Status] Needs Author Reply We would need you to make some changes or provide some more details about your PR. Thank you! OSS Citizen This Pull Request was opened by an Open Source contributor. labels May 11, 2023

jeherve requested changes May 12, 2023

View reviewed changes

gfog-floqast force-pushed the request_reviews branch 2 times, most recently from 2b5bbbb to a48acc5 Compare May 12, 2023 22:18

jeherve previously requested changes May 15, 2023

View reviewed changes

anomiex previously requested changes May 15, 2023

View reviewed changes

gfog-floqast commented May 23, 2023

View reviewed changes

gfog-floqast and others added 9 commits May 31, 2023 11:42

Adds feature where if review requirement not met, then request

f52d7e0

Adds changelog file

468f0db

updates for eslint

1365da9

removing updates to CHANGELOG.md

091853a

factors out getUsername and makes requests optional

b0c6c1f

Fix linting errors

b3d47e7

Adjust wording

0bdada8

adds missing request param check

539e82e

requesting the reviews from within the main function

a152832

refactor logic to main

f8dc253

gfog-floqast force-pushed the request_reviews branch from a9d73d4 to f8dc253 Compare May 31, 2023 19:09

anomiex reviewed Jun 2, 2023

View reviewed changes

gfog-floqast and others added 4 commits June 2, 2023 10:57

Update projects/github-actions/required-review/changelog/request-revi…

cdf5650

…ew-if-not-satisfied Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

Update projects/github-actions/required-review/src/request-review.js

717b41d

Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

Update projects/github-actions/required-review/src/requirement.js

c146f53

Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

addressing printSet and requirementsMet.length comments

264acb7

jeherve added [Status] Needs Review To request a review from Crew. Label will be renamed soon. and removed [Status] Needs Author Reply We would need you to make some changes or provide some more details about your PR. Thank you! labels Jun 7, 2023

Fix linting errors

e0e1082

anomiex reviewed Jun 8, 2023

View reviewed changes

projects/github-actions/required-review/src/request-review.js Outdated Show resolved Hide resolved

greg.fogelberg@floqast.com and others added 2 commits June 9, 2023 09:59

Expected '!=='

6b04ce2

Update projects/github-actions/required-review/src/request-review.js

49eeaa7

Co-authored-by: Brad Jorsch <anomiex@users.noreply.github.com>

anomiex approved these changes Jun 13, 2023

View reviewed changes

anomiex enabled auto-merge (squash) June 13, 2023 15:18

anomiex merged commit 9e6aa84 into Automattic:trunk Jun 13, 2023

github-actions bot removed the [Status] Needs Review To request a review from Crew. Label will be renamed soon. label Jun 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds feature where if review requirement not met, then request #30653

Adds feature where if review requirement not met, then request #30653

gfog-floqast commented May 11, 2023

github-actions bot commented May 11, 2023 •

edited

Loading

jeherve left a comment •

edited

Loading

jeherve May 12, 2023

jeherve May 12, 2023

jeherve May 12, 2023

gfog-floqast May 12, 2023

jeherve May 12, 2023

gfog-floqast commented May 12, 2023

jeherve left a comment

gfog-floqast commented May 15, 2023

jeherve commented May 15, 2023

gfog-floqast commented May 15, 2023

jeherve commented May 15, 2023

gfog-floqast commented May 15, 2023

anomiex left a comment

anomiex May 15, 2023 •

edited

Loading

gfog-floqast May 15, 2023

anomiex May 15, 2023

gfog-floqast May 15, 2023

anomiex May 15, 2023

anomiex May 15, 2023

gfog-floqast May 15, 2023

anomiex May 16, 2023

gfog-floqast commented May 23, 2023

gfog-floqast May 23, 2023

gfog-floqast commented May 23, 2023

anomiex commented May 23, 2023

gfog-floqast commented Jun 1, 2023

anomiex left a comment

gfog-floqast commented Jun 2, 2023

gfog-floqast commented Jun 7, 2023

gfog-floqast commented Jun 7, 2023

gfog-floqast commented Jun 9, 2023

anomiex left a comment

	} else {
	ok = false;
	core.endGroup();
	core.error( `Requirement "${ r.name }" is not satisfied by the existing reviews.` );
	}

	await reporter.status(
	core.getBooleanInput( 'fail' ) ? reporter.STATE_FAILURE : reporter.STATE_PENDING,
	reviewers.length ? 'Awaiting more reviews...' : 'Awaiting reviews...'
	);

		// Handle @singleuser virtual teams. Fetch the correct username case from GitHub
		// to avoid having to worry about edge cases and Unicode versions and such.

Adds feature where if review requirement not met, then request #30653

Adds feature where if review requirement not met, then request #30653

Conversation

gfog-floqast commented May 11, 2023

Proposed changes:

Other information:

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

github-actions bot commented May 11, 2023 • edited Loading

jeherve left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gfog-floqast commented May 12, 2023

jeherve left a comment

Choose a reason for hiding this comment

gfog-floqast commented May 15, 2023

jeherve commented May 15, 2023

gfog-floqast commented May 15, 2023

jeherve commented May 15, 2023

gfog-floqast commented May 15, 2023

anomiex left a comment

Choose a reason for hiding this comment

anomiex May 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gfog-floqast commented May 23, 2023

Choose a reason for hiding this comment

gfog-floqast commented May 23, 2023

anomiex commented May 23, 2023

gfog-floqast commented Jun 1, 2023

anomiex left a comment

Choose a reason for hiding this comment

gfog-floqast commented Jun 2, 2023

gfog-floqast commented Jun 7, 2023

gfog-floqast commented Jun 7, 2023

gfog-floqast commented Jun 9, 2023

anomiex left a comment

Choose a reason for hiding this comment

github-actions bot commented May 11, 2023 •

edited

Loading

jeherve left a comment •

edited

Loading

anomiex May 15, 2023 •

edited

Loading