Streamline and simplify v3 publishing job #8334
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change removes a bunch of cruft in the publishing job such that: - v2 is no longer supported, even incidentally (it's not been actually supported by Eng Services for some time, but the code has stuck around) - A single machine is used for publishing. This reduces the number of allocations, making it possible to move to 1ES pools (not in this change, yet). It also reduces general overhead associated with using multiple jobs and stages. Implementation is as follows: - Remove old, unused v3 publishing options and associated templates (nuget, source link and signing validation). - Collapse all publishing down into a single job that first looks up and validates the build and channel info, setting up any necessary variables for passing to the publish task. - Call the publish task with the associated parameters - Also fixes an issue where we were incorrectly passing IsInternalBuild. The value of that was being evaluated in the context of the arcade publishing job, not the actual build we were trying to promote. The value of this is just used for a safety check. - Remove publish installers and checksums options. This was hardcoded to true, and is legacy from when we were rolling this functionality out.
|
First official test (efcore -> general testing): https://dev.azure.com/dnceng/internal/_build/results?buildId=1547402&view=results Need to check a few others before I call it validated. |
|
For performance purposes, the average time for an efcore promotion appeared to be around 6:30ish. The tests show that it's about 2:30 with these changes. |
alexperovich
approved these changes
Jan 12, 2022
|
@mmitche heads up this seems to have broken publish using darc? https://dev.azure.com/dnceng/internal/_build/results?buildId=1549109&view=logs&j=226748d0-f812-5437-d3f0-2dd291f5666e&t=bad11196-972e-5d03-45a8-9db526506073 |
|
Yeah, some extra parameters that got removed. I'm going to revert this for now. I think it was just a testing gap, but want to make sure. |
mmitche
added a commit
that referenced
this pull request
Jan 12, 2022
mmitche
added a commit
that referenced
this pull request
Jan 12, 2022
mmitche
added a commit
that referenced
this pull request
Jan 13, 2022
mmitche
added a commit
to mmitche/arcade
that referenced
this pull request
Jan 28, 2022
) * Revert "Revert "Streamline and simplify v3 publishing job (dotnet#8334)" (dotnet#8339)" This reverts commit 7189b4a. * Remove parameter usage
mmitche
added a commit
that referenced
this pull request
Feb 2, 2022
* Revert "Revert "Streamline and simplify v3 publishing job"" (#8340) * Revert "Revert "Streamline and simplify v3 publishing job (#8334)" (#8339)" This reverts commit 7189b4a. * Remove parameter usage * Use 1ES pools in templates that run internally (#8344) * Use 1ES pools in templates that run internally - Use the 1ES pools in our templates, jobs, etc. - To reduce some of the cost of this, remove the "Setup Maestro Vars" job and replace with a step so that we don't end up with an additional allocation in those cases. - Call setup maestro vars to populate the azdo params in execute-sdl * Fix DevDiv promotion job (#8370) - Ensure devdiv promotion job uses devdiv pools - Use powershell core (the pools don't have IE initialized so we have to use basic parsing, which is default on powershell core) * Fixup common templates to be compatible with devdiv pools (#8371) In an earlier change, hosted pool usage was moved. This didn't account for differences in pools in devdiv and dnceng. * Fix oneloc template (#8381) The OneLoc template isn't used in the official build when not on main, so validation got skipped.
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change removes a bunch of cruft in the publishing job such that:
Implementation is as follows:
To double check: