cmd/go: runtime/debug build information not populated for test binaries when package name is not 'main'

[bcmills]

What did you do?

https://play.golang.org/p/0ITNcl4kXGN

$ gotip version
go version devel +8c5de667 Fri Aug 30 08:28:40 2019 +0000 linux/amd64

foo$ gotip mod init foo
go: creating new go.mod: module foo

foo$ cat > foo_test.go
package foo_test

import (
        "runtime/debug"
        "testing"
)

func TestBuildInfo(t *testing.T) {
        info, ok := debug.ReadBuildInfo()
        if !ok {
                t.Fatal("no debug info")
        }
        t.Log(info.Main.Version)
}

foo$ go test

What did you expect to see?

=== RUN   TestBuildInfo
--- PASS: TestBuildInfo (0.00s)
    foo_test.go:13: (devel)
PASS

(https://play.golang.org/p/0ITNcl4kXGN)

What did you see instead?

--- FAIL: TestBuildInfo (0.00s)
    foo_test.go:11: no debug info
FAIL

CC @jayconrod @rsc; see also #33975.

[breml]

I ended up on this issue while investigating how to access (non Go) files located in a sub-directory of my Go module. The access should happen from tests located in sub-sub-package.
We have it working with a relative path (e.g. ../../../folder/file) but it would be beneficial to have a less brittle way of accessing these files (especially for the case of refactoring).

So I tried the same as @bcmills is describing in this issue and I wanted to access info.Main.Path with the end goal of being able to access my files with info.Main.Path + "/folder/file".

Questions:

1. Is my assumption correct, that info.Main.Path would contain the basepath for the module, if the buildinfo would be available in test builds?
2. Why is the the buildinfo currently not available in test builds?

[breml]

Just as a note, using go list does work to get details about Go modules. E.g.:

func moduleBasePath(t *testing.T) string {
	t.Helper()

	err := os.Setenv("GO111MODULE", "on")
	if err != nil {
		t.Fatalf("unable to set GO111MODULE env var: %v", err)
	}

	cmd := exec.Command("go", "list", "-f", "{{.Module.Dir}}")
	out, err := cmd.Output()
	if err != nil {
		t.Fatalf("failed to evaluate Go module base path: %v", err)
	}

	return strings.TrimSpace(string(out))
}

[bcmills]

1. Is my assumption correct, that info.Main.Path would contain the basepath for the module, if the buildinfo would be available in test builds?

info.Main.Path in general contains a module path, not a filesystem path.

2. Why is the the buildinfo currently not available in test builds?

That is the bug described in this issue.

[bcmills]

Just as a note, using go list does work to get details about Go modules. E.g.:

Note that running go list in the working directory of a test will tell you about the module that contains the test itself, not the main module in which the test was built.

[breml]

1. Is my assumption correct, that info.Main.Path would contain the basepath for the module, if the buildinfo would be available in test builds?

info.Main.Path in general contains a module path, not a filesystem path.

OK, understood. So this would not have helped me anyway.

1. Why is the the buildinfo currently not available in test builds?

That is the bug described in this issue.

I was under the impression, that the buildinfo has been left out intentionally and not that it is a bug. Good to hear, that it is considered a bug.

[breml]

Just as a note, using go list does work to get details about Go modules. E.g.:

Note that running go list in the working directory of a test will tell you about the module that contains the test itself, not the main module in which the test was built.

That is not clear to me. Is this not the same in the case of tests? My understanding is, that go test builds for every package a test binary and executes it. The guarantee is, that the current working directory is actually the directory that contains the test. So therefore, the module that contains the test and the module the test was built for is the same (or is there something special going on in regards to modules when building tests?).
This is why go test -o test ./... does not work (error message: cannot use -o flag with multiple packages).

[bcmills]

The guarantee is, that the current working directory is actually the directory that contains the test.

Yes.

So therefore, the module that contains the test and the module the test was built for is the same (or is there something special going on in regards to modules when building tests?).

The module that contains the test is the module containing the working directory, yes.

However, that directory does not indicate the version of that module, and the versions reported by go list -m all will not be accurate: those versions are determined by the “main module” (in the sense of https://tip.golang.org/cmd/go/#hdr-The_main_module_and_the_build_list), not “the module containing package main”. (See #33975.)

[breml]

However, that directory does not indicate the version of that module, and the versions reported by go list -m all will not be accurate: those versions are determined by the “main module” (in the sense of https://tip.golang.org/cmd/go/#hdr-The_main_module_and_the_build_list), not “the module containing package main”. (See #33975.)

Thanks for your detailed explanation. This comment made it clear to me with the provided example. For my use case this does not matter, because the "main module" and "the module containing package main" are the same.

[ChrisHines]

I am exploring adding some logic and maybe a feature or two to github.com/go-stack/stack that take advantage of build information, but was surprised to discover that it didn't work with go test. I am glad to see it has already been reported, but sad that it hasn't been fixed yet.

I can probably get my ideas to work with a different testing strategy, but the new features would have to come with a big caveat about not working in tests since github.com/go-stack/stack is often used by logging code and that in turn is often relied upon to help diagnose test failures.

Has anyone investigated the level of effort to fix this? I am willing to help given some suggestions about where to start digging into the issue.

[ChrisHines]

I also have found that when the package is named main the BuildInfo.Path and .Main fields are populated but BuildInfo.Deps field is nil even when the package under test depends on code from other modules.

[mark-rushakoff]

I am working on a project that delivers a binary built with go test -c, and I was expecting to be able to embed and report the project's current commit with vcs.revision from debug.ReadBuildInfo().Settings.

I guess we can fall back to the old ldflags approach for my use case.

[dolanor]

This bug hit me as well as we use module info to start some server based on the availability on the system.
My alternative is to have a binary built externally, and run it with os.Exec, but it is not quite as clean as running the main func directly from the code.
Is there any idea on how this could be fixed?

[bcmills]

Is there any idea on how this could be fixed?

Probably the condition here needs to be updated:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/load/pkg.go;l=1965;drc=a6e5be0d30770e8bd1ba1e4bac2089218df121d9

I'm not sure exactly what to update it to, though.

[matloob]

We set the buildinfo for the testmain package using the buildinfo of the package under test: https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/load/test.go;l=284;drc=a85b8810c49515c469d265c399febfa48442a983. It seems to me like we can just call setBuildInfo on pmain (the testmain package) if the package under test does not have buildinfo set. That would be similar to what's being done in https://go.dev/cl/595961

[matloob]

cc @samthanawalla

[dekimsey]

I ran into this myself when I was working on a problem that required me to take advantage of Go's ability to create per-package test binaries to distribute tests to specific runtime/parallelize.

The resulting test binaries don't have any buildinfo, so it was not possible to interrogate the binaries themselves for their details either at runtime or externally with go version -m.

Context:
I had hoped to leverage the details to select certain packages to select packages of tests that needed to be executed with a different system user, root, but I didn't want to run the whole test suite as it took awhile and made re-combining test reports a wee bit ambiguous. The data would have been useful to know what the fully qualified path of the packgae originally came from. Between that and a precompiled test.pkg binary doesn't support go test's -json I think I just simply ran everything twice.

[matloob]

This also caused #69203. If we had populated build info on all test binaries, they'd have the default godebug incorporated from the build info into the action id.

[gopherbot]

Change https://go.dev/cl/613096 mentions this issue: cmd/go: populate build info for test mains

[gopherbot]

Change https://go.dev/cl/613275 mentions this issue: internal/upload: update test assertions to account for test build info