cmd/go: factor version reading into debug/buildinfo

[skelterjohn]

With go version 1.14 and go modules, we get a nice report from go version -m <binary>. The code backing this command is not available to use outside the binary, as it uses internal packages.

Supply chain analysis tools, which I work on, would benefit greatly from being able to reliably get this information using a package in the standard library. Instead, I'm copying and lightly modifying the code used to run the go version -m command since shelling out to a binary is not always an option.

[seankhliao]

To clarify, you're asking for a package to read the BuildInfo from a compiled executable?

[skelterjohn]

From the bytes of a compiled executable. Something like func ReadBuildInfo(io.Reader) (debug.BuildInfo, error) would be ideal.

[mvdan]

shelling out to a binary is not always an option

Can you clarify why? If you need to inspect Go binaries, having Go installed seems pretty reasonable.

[dmitshur]

Have you considered the rsc.io/goversion/version package?

[skelterjohn]

For why I prefer this to shelling out:

The product I work with is GCP's container image vulnerability scanner. It scans XX thousand container images per minute. Each of those container images has hundreds, thousands of files. I want to find Go binaries and report on how they were made so we can do vulnerability analysis, contributor history, etc. Shelling out to go version -m for each one is impractical for a number of reasons. First, in the execution environment I'm in (google's borg), starting a subprocess is an expensive operation that involves lots of extra logging. Second, the overhead to writing a file and running the go tool is far greater than inspecting the bytes as they fly by in our streaming read. Even if I limit it to executable files, there are still probably hundreds to inspect per image. Every cycle counts at this scale.

For consideration of rsc.io/goversion/version:

I did see that, and it does to approximately what I want. However, it does not have the interface I'd like (give it a file path rather than bytes, which increases the overhead per invocation significantly), and it's also not covered by any compatibility guarantees. For instance, the code it uses to load information from the executable and extract the mod string is different than that used by go version -m.

What I have done is copied, with light modifications to make it fit, the code that is used by go version -m to extract the mod string and the code used by runtime/debug.ReadBuildInfo() to parse it. This will work fine for now, but it implies an ongoing maintenance burden to ensure that it keeps up-to-date with any changes in the executable or mod string format. Since go version -m's code and runtime/debug.ReadBuildInfo()'s code imply that burden is being shouldered by the Go distribution as well, I would like to benefit :)

[jayconrod]

Thanks @skelterjohn for posting this. Vulnerability scanning is a use case we had in mind when embedding module version information in binaries. This would help complete the picture.

I think it would make sense to do this somewhere in x/mod, probably in a new package, since this doesn't fit in any existing package there.

cc @bcmills @FiloSottile @katiehockman @matloob @rsc

[rsc]

We discussed this some on #35667, where we decided it was OK to add go version -json to get a JSON-formatted output, and also OK to move the cmd/go version reader somewhere into x/mod. (Please use the one in cmd/go and not the rsc.io version, which is older.)

So this probably doesn't need to be a proposal, since we've already decided to do it.

[gopherbot]

Change https://golang.org/cl/348016 mentions this issue: buildinfo: add package for reading build metadata from Go executables

[gopherbot]

Change https://golang.org/cl/353887 mentions this issue: runtime/debug: add ReadBuildInfoFrom and ReadBuildInfoFromFile

[gopherbot]

Change https://golang.org/cl/356013 mentions this issue: doc/go1.18: add release notes for build and VCS info