cmd/go: most 'go mod' subcommands should not update go.mod, go.sum

[jayconrod]

As of Go 1.16, all go mod subcommands can update go.mod and go.sum. Some commands are intended to update these files, and that shouldn't change. However, most commands may still perform an automatic update, which may be surprising. Internally, go mod subcommands run in -mod=mod mode, even though they don't have a -mod command line flag.

For some commands, updating go.mod and go.sum is expected:

• go mod edit is supposed to edit go.mod.
• go mod init creates go.mod and may import information from another dependency management tool.
• go mod tidy is supposed to edit go.mod and go.sum.

For other commands, updating go.mod or go.sum may be quite unexpected. The go command should report an error if go.mod or go.sum need to be updated.

• go mod graph is a query tool. It shouldn't have side effects.
• go mod vendor is expected to update vendor but not go.mod.
• go mod verify also shouldn't have side effects.
• go mod why, same deal.

go mod download occupies an awkward middle ground. It has always been able to update go.mod (probably should not), but it may also add sums of modules in the build list to go.sum. This is usually unexpected and undesired when go mod download is invoked without arguments: users want something like go list all that populates the module cache with modules needed to build packages in the main module, not including unused dependencies of dependencies. However, we shouldn't entirely prevent go mod download from updating go.sum, since it's the best way to add a specific sum for a necessary but unused module (for disambiguating an imported package).

I'd suggest the following change in behavior for go mod download:

• go mod download always loads the build list and reports an error if go.mod needs to be updated instead of updating it automatically.
• go mod download only adds a hash to go.sum if the corresponding module is explicitly named on the command line (not simply matched by all or a wildcard or implied by a lack of arguments) without an explicit version or at the version in the build list.

cc @bcmills @matloob

[zikaeroh]

Out of curiosity, would this proposal change the behavior of go mod download when go.mod says one thing, but the code says another? I'm not personally clear what "build list" is defined as, but IIRC it may be determined by the code present in the module.

My main use case for go mod download is docker image caching, where I often do something like:

COPY ./go.mod ./go.sum ./
RUN go mod download

COPY ./ ./

RUN go build

Such that I can download the dependencies given in go.mod into a layer, then bring the code in; that way when I modify the code my module cache is preserved and I don't need to rebuild the image.

In this case, I cannot use go list all because that requires the code to be present, meaning that when the code is changed, docker invalidates the layer (as the files changed), and the module cache is gone.

[jayconrod]

@zikaeroh I think this proposal would improve things for you, at least a bit.

To clarify, go mod download doesn't read code, either before or after this proposal. When invoked without arguments or with the argument all, it downloads all the modules in the build list (the set of modules and versions selected by MVS). This is the same set of modules shown by go list -m all.

go mod download without arguments generally downloads modules you don't need (#41431). This proposal won't change that, but lazy loading (#36460) should help.

The difference here is that go mod download without arguments would not write go.sum, so you won't get a bunch unwanted changes there.

[zikaeroh]

To clarify, go mod download doesn't read code, either before or after this proposal. When invoked without arguments or with the argument all, it downloads all the modules in the build list (the set of modules and versions selected by MVS). This is the same set of modules shown by go list -m all.

Thanks for confirming. I'm alright with even a bad version of go mod download that over-downloads, since downloading some 2GB of dependencies once is better than redownloading 1GB every time I do a build. 🙂

[gopherbot]

Change https://golang.org/cl/310073 mentions this issue: cmd/go: explicitly set cfg.BuildMod in commands without -mod flag

[gopherbot]

Change https://golang.org/cl/330913 mentions this issue: cmd/go: call modload.WriteGoMod explicitly, once in each command

[gopherbot]

Change https://golang.org/cl/336151 mentions this issue: [dev.cmdgo] cmd/go: make fewer 'go mod' commands update go.mod

[gopherbot]

Change https://golang.org/cl/336152 mentions this issue: [dev.cmdgo] cmd/go: call modload.WriteGoMod explicitly, once in each command

[gopherbot]

Change https://golang.org/cl/341933 mentions this issue: cmd/go: make fewer 'go mod' commands update go.mod

[gopherbot]

Change https://golang.org/cl/347150 mentions this issue: cmd/go: call modload.WriteGoMod explicitly, once in each command

[gopherbot]

Change https://go.dev/cl/387920 mentions this issue: doc/go1.18: add release note for changes to automatic go.mod and go.sum updates