Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release and build pipelines #65

Merged
merged 51 commits into from
Sep 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
d7977fd
First attempt at build job
debonte Jun 22, 2024
db9fba9
Add TeamName var
debonte Jun 22, 2024
0768b5e
Add display names for script steps
debonte Jun 22, 2024
d6b775a
Try finding artifacts in $(Pipeline.Workspace)
debonte Jun 22, 2024
d158b77
Remove Pipeline.Workspace from output paths
debonte Jul 20, 2024
50ba92b
Add temporary ls step to try to locate wheel
debonte Jul 20, 2024
8d97344
Move output files to ArtifactStagingDirectory
debonte Jul 20, 2024
71c028b
Use var instead of wildcard when publishing
debonte Jul 20, 2024
4b5441a
Remove isOutput=true on releaseVersion
debonte Jul 20, 2024
70e0b60
Separate dirs for output files like pyrx to try to make sbom happy
debonte Jul 20, 2024
f29e49c
Add stages for creating GH release and waiting for validation
debonte Jul 20, 2024
f8baea5
Fix stage name
debonte Jul 20, 2024
fde624f
Validation should depend on CreateRelease not Build
debonte Jul 20, 2024
08112a3
Create release as a draft; add v prefix on release name
debonte Jul 20, 2024
2351f8d
Create release in my fork for now
debonte Jul 20, 2024
50e94ca
Create tag
debonte Jul 20, 2024
aac0e1d
Force ref to main branch
debonte Jul 20, 2024
cff8eed
Only trigger manually
debonte Jul 20, 2024
0ba448c
Add main branch to git tag command
debonte Jul 20, 2024
826affe
Set ;isOutput=true on releaseVersionWithPrefix
debonte Jul 20, 2024
12717d1
Set git user email and name when pushing tag
debonte Jul 20, 2024
b2ddc02
Try fix tag creation with persistCredentials
debonte Sep 17, 2024
033ea33
TEMP: Create GH release in fork
debonte Sep 17, 2024
92a9263
Add pr: none
debonte Sep 17, 2024
a8c3370
Add publishing stage
debonte Sep 17, 2024
67e4b83
Separate out build; add signing
debonte Sep 17, 2024
b6ea3a7
Remove signing
debonte Sep 17, 2024
9e2757c
Move ARTIFACT_NAME_FOO vars to build.yml
debonte Sep 17, 2024
9fbbb4c
Specify `tag` instead of `tagSource`
debonte Sep 17, 2024
25313c6
Try to use built-in `download` task
debonte Sep 17, 2024
0fe6273
Extract build stage into template
debonte Sep 17, 2024
a02ea49
Try relative template paths
debonte Sep 17, 2024
0c21ad3
Add stages: at top of build_stage.yml
debonte Sep 17, 2024
b444616
Update artifact paths to `dist`
debonte Sep 17, 2024
58b0318
tagSource: userSpecifiedTag
debonte Sep 18, 2024
fa626a6
Use main as target instead of tag
debonte Sep 18, 2024
2ddaf6e
Try to fix release title and assets
debonte Sep 18, 2024
f581403
Try to share ARTIFACT_NAME_WHEEL via globals.yml
debonte Sep 18, 2024
615f9cb
Try output dir and input instead of download
debonte Sep 18, 2024
e3e505c
Try releaseJob
debonte Sep 18, 2024
e33010d
isProduction: true
debonte Sep 18, 2024
f5e6d61
Move checkout out of releaseJob
debonte Sep 18, 2024
34433ea
Add Build as CreateRelease dependency for vars
debonte Sep 18, 2024
6a603f1
Make publishing step a releaseJob
debonte Sep 18, 2024
927acee
Switch back to MicroBuild.1ES.Official.yml
debonte Sep 18, 2024
8129a7d
Should tag source branch since that's what is built
debonte Sep 18, 2024
617916e
Create releases in microsoft/sarif-tools not my fork
debonte Sep 18, 2024
b88849b
Add displayName for CreateTag
debonte Sep 18, 2024
7280c23
Move architecture variable from release.yml to build_stage.yml
debonte Sep 18, 2024
e59ddfd
Add blank line at end of build_stage
debonte Sep 18, 2024
875710e
Use build pipeline only for CI builds
debonte Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 31 additions & 7 deletions azure-pipelines/build.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,34 @@
name: Build
trigger: none
pr: none
trigger:
branches:
include:
- main
paths:
exclude:
- azure-pipelines/release.yml

stages:
- stage: A
jobs:
- job: A1
steps:
- bash: echo "Hello world"
resources:
repositories:
- repository: 1ESPipelineTemplates
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release

variables:
TeamName: sarif-tools

extends:
template: v1/1ES.Official.PipelineTemplate.yml@1ESPipelineTemplates
parameters:
sdl:
sourceAnalysisPool: VSEngSS-MicroBuild2022-1ES
pool:
name: AzurePipelines-EO
demands:
- ImageOverride -equals 1ESPT-Ubuntu22.04
os: Linux
customBuildTags:
- ES365AIMigrationTooling
stages:
- template: templates/build_stage.yml@self
128 changes: 122 additions & 6 deletions azure-pipelines/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,125 @@ name: Release
trigger: none
pr: none

stages:
- stage: A
jobs:
- job: A1
steps:
- bash: echo "Hello world"
variables:
- template: templates/globals.yml
- name: TeamName
value: sarif-tools

resources:
repositories:
- repository: MicroBuildTemplate
type: git
name: 1ESPipelineTemplates/MicroBuildTemplate
ref: refs/tags/release

extends:
template: azure-pipelines/MicroBuild.1ES.Official.yml@MicroBuildTemplate
parameters:
sdl:
sourceAnalysisPool: VSEngSS-MicroBuild2022-1ES
pool:
name: AzurePipelines-EO
demands:
- ImageOverride -equals 1ESPT-Ubuntu22.04
os: Linux
customBuildTags:
- ES365AIMigrationTooling
stages:
- template: templates/build_stage.yml@self

- stage: CreateTag
displayName: Create Tag
dependsOn: Build
variables:
releaseVersionWithPrefix: $[ stageDependencies.Build.Build.outputs['getReleaseVersionStep.releaseVersionWithPrefix'] ]
jobs:
- job: CreateTag
steps:
- checkout: self
fetchDepth: 1
fetchTags: false
persistCredentials: true

- script: |
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config user.name "Azure Piplines"
git fetch --depth 1 origin $(Build.SourceBranchName)
git tag -a $(releaseVersionWithPrefix) -m "Release $(releaseVersionWithPrefix)" origin/$(Build.SourceBranchName)
git push origin $(releaseVersionWithPrefix)
displayName: Create git tag

- stage: CreateRelease
displayName: Create GitHub Release
dependsOn:
- Build
- CreateTag
variables:
releaseVersionWithPrefix: $[ stageDependencies.Build.Build.outputs['getReleaseVersionStep.releaseVersionWithPrefix'] ]
jobs:
- job: CreateRelease
templateContext:
type: releaseJob
isProduction: true
inputs:
- input: pipelineArtifact
artifactName: $(ARTIFACT_NAME_WHEEL)
targetPath: $(Build.StagingDirectory)/dist
steps:
- task: GitHubRelease@1 #https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/github-release-v1?view=azure-pipelines
displayName: Create GitHub Release
inputs:
gitHubConnection: GitHub-sarif-tools
repositoryName: microsoft/sarif-tools
action: create
target: $(Build.SourceBranchName)
title: $(releaseVersionWithPrefix)
tag: $(releaseVersionWithPrefix)
tagSource: userSpecifiedTag
isDraft: true
addChangeLog: false
assets: $(Build.StagingDirectory)/dist/*

- stage: WaitForValidation
dependsOn: CreateRelease
jobs:
- job: wait_for_validation
displayName: Wait for manual validation
pool: server
steps:
- task: ManualValidation@0
timeoutInMinutes: 1440 # task times out in 1 day
inputs:
notifyUsers: erikd@microsoft.com
instructions: Please test the latest draft release and then publish it.
onTimeout: reject

- stage: Release
dependsOn: WaitForValidation
jobs:
- job: PublishToPyPi
displayName: Release to PyPi

pool:
name: VSEngSS-MicroBuild2022-1ES # This pool is required to have the certs needed to publish to PyPi using ESRP.
os: windows
image: server2022-microbuildVS2022-1es

templateContext:
type: releaseJob
isProduction: true
inputs:
- input: pipelineArtifact
artifactName: $(ARTIFACT_NAME_WHEEL)
targetPath: $(Build.StagingDirectory)/dist

steps:
- template: MicroBuild.Publish.yml@MicroBuildTemplate
parameters:
intent: PackageDistribution
contentType: PyPi
contentSource: Folder
folderLocation: $(Build.StagingDirectory)/dist
waitForReleaseCompletion: true
owners: erikd@microsoft.com
approvers: grwheele@microsoft.com
43 changes: 43 additions & 0 deletions azure-pipelines/templates/build_stage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
stages:
- stage: Build
variables:
- template: globals.yml
jobs:
- job: Build

templateContext:
outputs:
- output: pipelineArtifact
targetPath: $(Build.StagingDirectory)/dist
sbomBuildDropPath: $(Build.StagingDirectory)/dist
artifactName: $(ARTIFACT_NAME_WHEEL)

variables:
python.version: "3.8"
architecture: x64

steps:
- template: use_python.yml@self

- script: pipx install poetry
displayName: Install Poetry

- script: poetry build --no-interaction
displayName: poetry build

- powershell: |
$releaseVersion = & poetry version --short
echo "releaseVersion: $releaseVersion"
echo "##vso[task.setvariable variable=releaseVersion]$releaseVersion"
echo "##vso[task.setvariable variable=releaseVersionWithPrefix;isOutput=true]v$releaseVersion"
displayName: Get release version
name: getReleaseVersionStep

- task: CopyFiles@2
displayName: Copy wheel and tarball
inputs:
sourceFolder: dist
targetFolder: $(Build.StagingDirectory)/dist
contents: |
sarif_tools-$(releaseVersion)-py3-none-any.whl
sarif_tools-$(releaseVersion).tar.gz
2 changes: 2 additions & 0 deletions azure-pipelines/templates/globals.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
variables:
ARTIFACT_NAME_WHEEL: wheel
6 changes: 6 additions & 0 deletions azure-pipelines/templates/use_python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: "$(python.version)"
architecture: "$(architecture)"
displayName: "Use Python $(python.version) $(architecture)"