cmd/doc: 'go doc $pkg' prefers GOPATH instead of active module

[bcmills]

go doc calls out to cmd/doc, and cmd/doc implicitly uses GOPATH to resolve the imports instead of looking in the active modules. That results in unexpected skew: it produces spurious failures when the package exists but is not in GOPATH, and spurious documentation if GOPATH contains unpublished changes (as in the example in the details below).

This is one instance of #24661, but the tool in question happens to be bundled with cmd/go and released with the Go toolchain: we should fix it sooner rather than later.

CC @robpike @mvdan @ianthehat @rsc

~/src/golang.org/x/build$ export GO111MODULE=on

~/src/golang.org/x/build$ go version
go version devel +d8b7a9286d Wed Nov 28 13:07:32 2018 -0500 linux/amd64

~/src/golang.org/x/build$ go list -m golang.org/x/tools
golang.org/x/tools v0.0.0-20181030000716-a0a13e073c7b

~/src/golang.org/x/build$ go list golang.org/x/tools/cmd/goforward
go: downloading golang.org/x/tools v0.0.0-20181030000716-a0a13e073c7b
go: extracting golang.org/x/tools v0.0.0-20181030000716-a0a13e073c7b
go: finding golang.org/x/tools/cmd/goforward latest
go: finding golang.org/x/tools/cmd latest
go: finding golang.org/x/tools latest
go: downloading golang.org/x/tools v0.0.0-20181127232545-e782529d0ddd
go: extracting golang.org/x/tools v0.0.0-20181127232545-e782529d0ddd
can't load package: package golang.org/x/tools/cmd/goforward: unknown import path "golang.org/x/tools/cmd/goforward": cannot find module providing package golang.org/x/tools/cmd/goforward

~/src/golang.org/x/build$ go doc golang.org/x/tools/cmd/goforward | head -n 2
The goforward command adds forwarding declarations from one Go package to
another.

[mvdan]

I was under the impression that cmd/doc had been adapted to modules with the rest of the built-in tooling. Thanks for raising the issue.

I presume that we should keep the GOPATH logic for when the command is called outside a module, and introduce module logic otherwise.

I believe that only dirs.go would need changes, as that's the part that scans GOPATH and feeds the packages it finds to the rest of the code.

[bcmills]

I presume that we should keep the GOPATH logic for when the command is called outside a module, and introduce module logic otherwise.

No, we should only use the GOPATH logic when not in module mode (that is, when go env GOMOD produces the empty string).

In module mode outside of a module (as of CL 148517) we should produce documentation for the latest version of the package, perhaps using go list and/or go mod download.

[mvdan]

@myitcv rightfully pointed out https://go-review.googlesource.com/c/go/+/126799 - it does seem like support was added.

[bcmills]

Ah, looks like the bug is here:

go/src/cmd/doc/dirs.go

Line 165 in 96d4178

usingModules = bytes.Contains(stdout, []byte("go.mod"))

I'm setting GOMOD to os.DevNull in module mode outside a module. Might be a trivial fix — thanks for the reference!

[bcmills]

Turns out not to be so trivial after all. CL 126799 added module-mode support for everything except packages, which apparently need more than just the source code for the package (i.e. need go/packages).

CC @matloob

[gopherbot]

Change https://golang.org/cl/151617 mentions this issue: cmd/doc: treat any non-empty GOMOD as module mode

[bcmills]

CC @ianthehat @dmitshur

[dmitshur]

I tested with the following steps, using Go 1.12.4:

$ export GO111MODULE=on
$ export GOPATH="$(mktemp -d)"
$ cd $(mktemp -d)
$ go mod init m
$ go get rsc.io/quote/v3
$ go get dmitri.shuralyov.com/test/modtest1/inner/p@v1.0.0

$ go doc rsc.io/quote/v3
$ go doc dmitri.shuralyov.com/test/modtest1/inner/p
# worked as expected

$ mkdir -p $GOPATH/src/rsc.io/quote/v3 $GOPATH/src/dmitri.shuralyov.com/test/modtest1/inner/p
$ echo 'package p; const Foo = "bar"' > $GOPATH/src/rsc.io/quote/v3/p.go
$ echo 'package p; const Foo = "bar"' > $GOPATH/src/dmitri.shuralyov.com/test/modtest1/inner/p/p.go

$ go doc rsc.io/quote/v3
$ go doc dmitri.shuralyov.com/test/modtest1/inner/p
# incorrectly displays documentation for package from GOPATH rather than in module cache

It seems that go doc works as expected in module mode when a package is available only in the local module cache and not in a GOPATH workspace.

There is an issue of go doc, when in module mode, still preferring a package in GOPATH workspace when it's available in both.

[bcmills]

@dmitshur, is this still in progress? (If so, are you still planning to fix it for 1.13, or should we move it to the 1.14 milestone?)

[dmitshur]

This is in progress, I am investigating it now.

If the bug fix ends up being reasonable, it can still go into 1.13. If it turns out to be very complicated and risky, we can consider pushing it to 1.14.

[dmitshur]

The problem was that a few build.Import calls did not provide the current working directory, which is a hard requirement for finding packages in module mode. So the fix is small. I sent CL 183991.

[gopherbot]

Change https://golang.org/cl/183991 mentions this issue: cmd/doc: provide working directory to build.Import calls