Backport of build: Further polishing of the prototype build workflow into v1.2 #31115
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
teamterraform
force-pushed
the
backport/build-workflow-dev/general-polish/eagerly-natural-aardvark
branch
from
7614a8c to
401468f
Compare
The actions/checkout workflow does some heroics to try to fetch exactly the commit being built and nothing else, even if asked to fetch the history leading up to that commit. That means we don't end up having enough information to get an accurate answer from "git describe". Since we're intentionally relying on the git history here, we'll fetch all of the tags explicitly after initial checkout. Although that does add some delay to this step, we're intentionally doing this version calculation only once as a separate workflow job so that all of the other jobs can still benefit from this action's quicker checkout behavior.
This also sets an additional variable if it detects that this is an alpha or development build, which currently does nothing but might eventually turn on the ability to use experimental features, if we make that something available only in prereleases.
Previously we'd discard everything after the second dash.
apparentlymart
force-pushed
the
backport/build-workflow-dev/general-polish/eagerly-natural-aardvark
branch
from
401468f to
643c7dc
Compare
apparentlymart
deleted the
backport/build-workflow-dev/general-polish/eagerly-natural-aardvark
branch
|
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This PR is auto-generated from #31099 to be assessed for backporting due to the inclusion of the label 1.2-backport.
The below text is copied from the body of the original PR.
Back in #30283 we introduced a prototype new build process that for now just signals whether we were able to build working executables for various platforms, but is hopefully destined to one day become the first step in our routine release process, producing the artifacts that we'll ultimately publish.
We merged that PR early in order to get some experience with its behavior over several different kinds of releases, since this sort of thing is notoriously hard to test realistically in an artificial environment. Since then, we've learned about a couple of quirks that this PR aims to now address:
"Build Terraform Packages" GitHub Action not quite right with its tag-based version detection #30864: Due to how the standard "checkout" GitHub Action tries to fetch as little as possible, it was not producing a fully-fledged-enough repository for us to analyze tags to automatically determine a version number. Consequently, it ended up inconsistently misclassifying real releases as being tagged ones.
That's fixed here by retroactively "unshallowing" the repository and explicitly fetching all of the tags. That then gives us sufficient data for
git describeto return a correct result. If the commit being built exactly matches a tag then the version number will be derived from that tag, and if not it will derive a "close enough" modified version number by suffixing a number of commits since the tag and an abbeviated commit ID of the selected commit.In A new build process, based on GitHub Actions #30283 I'd intentionally skipped having the build process write the detected version number into the built executables because I was anticipating that cmd/go: stamp git/vcs current HEAD hash/commit hash/dirty bit in binaries golang/go#37475 would cause the Go toolchain to make that information available in a different way after Go 1.18. However, in subsequent testing I learned that it currently stamps only the commit ID and not any module version information for the main package. cmd/go: stamp the pseudo-version in builds generated by
go buildgolang/go#50603 is an open request tracking the behavior I had hoped for, but it remains unimplemented at the time I'm writing this and there are some doubts about what exactly it would do if it were implemented.In light of that, I've instead adopted the typical old strategy of using linker arguments to write dynamically-selected strings into the executable at build time. This achieves the desired effect of making
go versionreturn the intended version number.In addition to these two quirks, I've also been continuing to think about what I proposed in #30948 and as part of that I wanted to explore what it might look like for the build process to be responsible for annotating builds as allowing experiments. At the moment that doesn't actually have any effect, since #30948 is not merged. Continuing with the previous goal of using this as an experimental vehicle to see how this build process treats different releases we'll run in the coming months, I'd like to include that here only so I can inspect the real builds that will follow and make sure this workflow makes the correct selections for which builds ought to have experiment support.
I'm proposing also to backport this to the
v1.2branch so that we can observe its behavior with the v1.2.x patch releases, since it is likely to be at least several months before we're running any non-prerelease releases from themainbranch and I want to observe how it treats both real releases and alpha releases.Because this PR is from a branch that has the prefix
build-workflow-dev/, you can see the results of running this workflow as checks on this PR. (In the normal case we only run this workflow after merge, in the context of a real release branch, in order to keep the PR check turnaround time reasonable and to keep resource usage reasonable.)Since this workflow currently doesn't feed into anything downstream, the risk of changing this is pretty low: in the worst case, we might see some failing runs of this workflow after a PR is merged or after a release tag is created by our current process. Such errors will not block any other processes and so I will attend to them asynchronously as needed.