Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pipeline configuration for cleaning up upstream branches #1088

Merged
4 commits merged into from
Oct 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions eng/common/scripts/Delete-RemoteBranches.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
param(
$RepoOwner,
$RepoName,
$BranchPrefix,
$AuthToken
)

. "${PSScriptRoot}\common.ps1"

LogDebug "Operating on Repo [ $RepoName ]"
try{
$branches = (List-References -RepoOwner $RepoOwner -RepoName $RepoName -Ref "heads/$BranchPrefix").ref
}
catch {
LogError "List-References failed with exception:`n$_"
exit 1
}

foreach ($branch in $branches)
{
try {
$branchName = $branch.Replace("refs/heads/","")
$head = "${RepoOwner}/${RepoName}:${branchName}"
LogDebug "Operating on branch [ $branchName ]"
$pullRequests = List-PullRequests -RepoOwner $RepoOwner -RepoName $RepoName -head $head
}
catch
{
LogError "List-PullRequests failed with exception:`n$_"
exit 1
}

if ($pullRequests.Count -eq 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we log cases where we find branches that have a pull request and list the branch and what open pull requests there are still for it?

{
LogDebug "Branch [ $branchName ] in repo [ $RepoName ] has no associated Pull Request. Deleting Branch"
try{
Delete-References -RepoOwner $RepoOwner -RepoName $RepoName -Ref ($branch.Remove(0,5)) -AuthToken $AuthToken
mitchdenny marked this conversation as resolved.
Show resolved Hide resolved
}
catch {
LogError "Delete-References failed with exception:`n$_"
exit 1
}
}
}
62 changes: 62 additions & 0 deletions eng/common/scripts/Invoke-GitHubAPI.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,24 @@ function Invoke-GitHubAPIPatch {
return $resp
}

function Invoke-GitHubAPIDelete {
param (
[Parameter(Mandatory = $true)]
$apiURI,
[Parameter(Mandatory = $true)]
$token
)

$resp = Invoke-RestMethod `
-Method DELETE `
-Uri $apiURI `
-Headers (Get-GitHubHeaders -token $token) `
-MaximumRetryCount 3

return $resp
}


function Invoke-GitHubAPIGet {
param (
[Parameter(Mandatory = $true)]
Expand Down Expand Up @@ -105,6 +123,27 @@ function List-PullRequests {
return Invoke-GitHubAPIGet -apiURI $uri
}

#
<#
.PARAMETER Ref
Ref to search for
Pass 'heads/<branchame> ,tags/<tag name>, or nothing
#>
function List-References {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have a better name for this function as List-References without context is hard to understand as it is very ambiguous with other things. For example reference means something in PS as well. Perhaps List-GithubSourceReferences or something like that might be a little clearer.

param (
[Parameter(Mandatory = $true)]
$RepoOwner,
[Parameter(Mandatory = $true)]
$RepoName,
$Ref
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point we might also need to pass a token here if we ever call this on a private repo for example.

)

$uri = "$GithubAPIBaseURI/$RepoOwner/$RepoName/git/matching-refs/"
if ($Ref) { $uri += "$Ref" }

return Invoke-GitHubAPIGet -apiURI $uri
}

function Add-IssueComment {
param (
[Parameter(Mandatory = $true)]
Expand Down Expand Up @@ -229,4 +268,27 @@ function Update-Issue {
}

return Invoke-GitHubAPIPatch -apiURI $uri -body $parameters -token $AuthToken
}

function Delete-References {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a better name would be Delete-GithubSourceReference.

param (
[Parameter(Mandatory = $true)]
$RepoOwner,
[Parameter(Mandatory = $true)]
$RepoName,
[ValidateNotNullOrEmpty()]
[Parameter(Mandatory = $true)]
$Ref,
[Parameter(Mandatory = $true)]
$AuthToken
)

if ($Ref.Trim().Length -eq 0)
{
throw "You must supply a valid 'Ref' Parameter to 'Delete-Reference'."
}

$uri = "$GithubAPIBaseURI/$RepoOwner/$RepoName/git/refs/$Ref"

return Invoke-GitHubAPIDelete -apiURI $uri -token $AuthToken
}
32 changes: 32 additions & 0 deletions eng/pipelines/templates/jobs/tools-cleanup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
parameters:
- name: Repos
type: object
default:
- azure-sdk-for-android
- azure-sdk-for-c
- azure-sdk-for-cpp
- azure-sdk-for-go
- azure-sdk-for-ios
- azure-sdk-for-java
- azure-sdk-for-js
- azure-sdk-for-net
- azure-sdk-for-python

jobs:
- job: CleanUp
pool:
vmImage: windows-2019
steps:
- ${{ each repo in parameters.Repos }}:
- task: PowerShell@2
displayName: Clean Up Sync Common Branches
condition: succeeded()
inputs:
pwsh: true
workingDirectory: $(System.DefaultWorkingDirectory)
filePath: $(System.DefaultWorkingDirectory)/eng/common/scripts/Delete-RemoteBranches.ps1
arguments: >
-RepoOwner "Azure"
-RepoName ${{ repo }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you are not passing RepoOwner. I assume you plan to pass azure-sdk as these are cleaning up branches in the forks.

We should also consider doing a similar branch clean-up for the increment version branches.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually , This is for cleaning up upstream branches pushed to the main repos as part of the new process. So RepoOwner is azure. I was defaulting it in the script earlier.

Do we need to do cleanup on the 'azure-sdk' owned repos?

-BranchPrefix "sync-eng/common-"
-AuthToken $(azuresdk-github-pat)