cmd/go: go mod -sync fails although go get succeeds

[rogpeppe]

go version devel +5f5402b Tue Jul 24 23:54:08 2018 +0000 linux/amd64
git version 2.18.0

What did you do?

I created a module, did a go get at a particular version, then updated it with another go get, and then ran go mod -sync to try to remove the spurious dependencies. It failed unexpectedly. I then ran the command again with some log statements added in the Import function in src/cmd/go/internal/modload/import.go.

Given that the go get command worked fine, I'd expect go mod -sync to work OK too.

% mkdir /tmp/modtest
% cd /tmp/modtest
% go mod -init -module example.com/example
go: creating new go.mod: module example.com/example
% lc
go.mod
% cat > x.go
package example
import _ "github.com/juju/worker.v1"
% go get gopkg.in/juju/worker.v1@v1.0.0-20161003161701-8b18096b52dc
go: finding gopkg.in/tomb.v1 latest
go: finding github.com/juju/errors latest
go: finding github.com/juju/loggo latest
go: downloading github.com/juju/loggo v0.0.0-20180524022052-584905176618
% cat go.mod
module example.com/example

require (
	github.com/juju/errors v0.0.0-20170703010042-c7d06af17c68 // indirect
	github.com/juju/loggo v0.0.0-20180524022052-584905176618 // indirect
	gopkg.in/juju/worker.v1 v1.0.0-20161003161701-8b18096b52dc // indirect
	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
)
% go get gopkg.in/juju/worker.v1
go: finding gopkg.in/juju/worker.v1 latest
go: finding gopkg.in/tomb.v2 latest
go: finding github.com/juju/utils/clock latest
go: finding github.com/juju/utils latest
go: downloading github.com/juju/utils v0.0.0-20180619112806-c746c6e86f4f
% cat go.mod
module example.com/example

require (
	github.com/juju/errors v0.0.0-20170703010042-c7d06af17c68 // indirect
	github.com/juju/loggo v0.0.0-20180524022052-584905176618 // indirect
	github.com/juju/utils v0.0.0-20180619112806-c746c6e86f4f // indirect
	gopkg.in/juju/worker.v1 v1.0.0-20171206115140-5662d86e8af0 // indirect
	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
	gopkg.in/tomb.v2 v2.0.0-20161208151619-d5d1b5820637 // indirect
)
% go mod -sync
go: import "example.com/example" ->
	import "github.com/juju/worker.v1": cannot find module providing package github.com/juju/worker.v1
% go mod -v -sync
trying path "github.com/juju/worker.v1"
query failed: git ls-remote -q https://github.com/juju/worker.v1 in /home/rog/src/go/src/mod/cache/vcs/831b3cb88bfefae5e3e15cecfa319ab143baa9e949c11796e7e8f9e86b102ebf: exit status 128:
	ERROR: Repository not found.
	fatal: Could not read from remote repository.
	
	Please make sure you have the correct access rights
	and the repository exists.
trying path "github.com/juju"
query failed: invalid github.com/ import path "github.com/juju"
trying path "github.com"
query failed: unrecognized import path "github.com" (parse https://github.com?go-get=1: no go-import meta tags ())
go: import "example.com/example" ->
	import "github.com/juju/worker.v1": cannot find module providing package github.com/juju/worker.v1

[rogpeppe]

Another potentially interesting aspect of the above log is that gopkg.in/juju/worker.v1 is marked as indirect in go.mod after the first go get when it's actually mentioned specifically in the source code.

[bcmills]

Possibly the same underlying cause as #26602.

[rsc]

"go get" downloads and builds the packages you named, just as it always has.

"go mod -sync" makes sure that everything in the main (current) module and its dependencies is buildable.

Note that typically you shouldn't need "go get" - just add the import and run "go build".
In this case plain "go build" would have failed just like "go mod -sync" did.

You did not remark on the fact that x.go imports a path from github.com instead of gopkg.in. Was that intentional?
(I can see that if you thought x.go said gopkg.in, this transcript would be a lot more mysterious.)

It's possible that "go get" should do more, but I am not sure how much flexibility we really have here.
If I do "go get rsc.io/2fa" to get that binary and there is some irrelevant unbuildable package in rsc.io/2fa/buggy,
the "go get" does not fail today and probably should continue not to fail.

[rogpeppe]

You did not remark on the fact that x.go imports a path from github.com instead of gopkg.in. Was that intentional?
(I can see that if you thought x.go said gopkg.in, this transcript would be a lot more mysterious.)

Yes, you're right - I had intended to write "gopkg.in", and that explains this behaviour.
I think that (again) I had difficulty diagnosing the situation correctly because of the opaque
error ("cannot find module providing package") which covers many possible error causes.

I'll close this issue as invalid. Sorry for the noise.