cmd/go: go directive is insufficiently documented for module authors to be able to make a decision about its value

[dmitshur]

Go 1.12 has added a go directive to the go.mod file. It's documented in the following places.

At https://golang.org/doc/go1.12#modules:

The go directive in a go.mod file now indicates the version of the language used by the files within that module. It will be set to the current release (go 1.12) if no existing version is present. If the go directive for a module specifies a version newer than the toolchain in use, the go command will attempt to build the packages regardless, and will note the mismatch only if that build fails.

Additionally, a relevant paragraph from https://golang.org/doc/go1.12#compiler:

The compiler now accepts a -lang flag to set the Go language version to use. For example, -lang=go1.8 causes the compiler to emit an error if the program uses type aliases, which were added in Go 1.9. Language changes made before Go 1.12 are not consistently enforced.

At https://golang.org/cmd/go/#hdr-The_go_mod_file:

go, to set the expected language version;

There is additional information available in issues, proposals, commit messages and code review comments, which I did not mention above. However, that information is not readily accessible to users.

It is very common for package authors to aim to ensure their Go packages can be used in multiple versions of Go. Sometimes it's just Go 1.12.x and Go 1.11.x (the current and previous releases). Other times the goal is to support even older versions of Go. This often includes running tests in CI with those versions of Go to ensure that the build and tests are successful. When these package authors add a go.mod file file to their repositories, they should be able to make a sensible decision about what go directive should be included.

Problem

I believe the current documentation is not sufficient for module authors to make a well-informed decision on whether to include the go directive in their go.mod files, and if so, what the value of the go directive should be.

I think we should try to improve documentation to resolve that. But first I want to make sure others agree that it's a problem.

(This is based on looking over the discussions that have happened around various PRs/CLs where the go directive is introduced, often by people who are very familiar with modules. Sometimes this happens more often due to #30790.)

/cc @ianlancetaylor @dsnet @bcmills @julieqiu @heschik @matloob

[ianlancetaylor]

I'm sure we should write better docs.

The basic guideline is fairly simple: a "go" directive 1.N means that the code in that module is permitted to use language features that existed in 1.N even if those features were removed in later releases of the language.

In other words, if the code compiles successfully at version 1.N, then it will continue to compile for all later versions of the language, even if it happens to use language features that were later removed.

To a first approximation, nobody should ever have to worry about the "go" directive. The only likely time you might need to set it manually is if you are copying some existing code to a new module, that code uses some obsolete language feature, and you don't have time to rewrite it. In that case you might want to set the "go" directive (using go mod edit -go) to the version used by the original module, thus permitting your new module to use the obsolete features. When writing new code you will presumably simply avoid the obsolete language features.

[bcmills]

a "go" directive 1.N means that the code in that module is permitted to use language features that existed in 1.N even if those features were removed in later releases of the language.

My understanding of #28221 is that it should not only permit the older features, but also prohibit newer features. Otherwise, there is little incentive for folks to upgrade to a language version beyond the removed features: they can just set go 1.11 and get every feature.

That understanding is based on this part of the proposal discussion:

suppose we had this feature for the 1.9 release, when type aliases were introduced to the language. Then passing -lang=go1.8 to the compiler would give a syntax error for any code that used a type alias.

That has an interesting interaction with build tags, though: if I set go 1.11 in my go.mod file, but then have a source file somewhere that specifies // +build go1.13, is that file permitted to use language features that were added in 1.13? Is it permitted to use language features that were removed in 1.13? Or should that file be ignored entirely, since the go 1.11 language did not include a go1.13 tag by default?

[bcmills]

I would argue that a // +build go1.13 file in a go 1.11 module should be ignored, since it would have been ignored when the module was originally tested under Go 1.11.

Or perhaps it should result in an explicit error, since the user may otherwise be confused that it has no effect.

[bcmills]

(CC @jayconrod)

[dmitshur]

Those are good questions. When looking for an answer, we should make sure that authors who wish to create Go modules that work with a wide range of Go versions (e.g., 1.8–1.12 as go-cmp does, 1.2–1.12 as bluemonday does, or all versions of Go as iter does) have a good way of selecting the value of the go directive. /cc @dsnet @buro9 @bradfitz

[dmitshur]

That understanding is based on this part of the proposal discussion:

suppose we had this feature for the 1.9 release, when type aliases were introduced to the language. Then passing -lang=go1.8 to the compiler would give a syntax error for any code that used a type alias.

I made similar conclusions based on that type alias example. When I asked Ian about it, his response was:

When I added the -lang option to cmd/compile, I had to have some way
to test the way the go tool handled it. So, I changed cmd/compile to
reject type aliases when using a -lang option before Go 1.9. But for
future backward-compatible changes, like for example the support for
0o number prefixes that we just added, I do not expect that
cmd/compile will be changed to reject the new features when building
for an older version.

[...] it is not intended to be taken as guidance for future work.

Which made me understand that the type alias example can be misleading when thinking about how the go directive value should be chosen.

[rsc]

I will write some docs.

[rsc]

See #31747 too.

[kortschak]

I would argue that a // +build go1.13 file in a go 1.11 module should be ignored, since it would have been ignored when the module was originally tested under Go 1.11.

Or perhaps it should result in an explicit error, since the user may otherwise be confused that it has no effect.

On the basis of this, it seems like an odd behaviour for go test to attempt to mutate the go.mod file if there is no go directive line. If I have a go mod file that makes no claims about the language version, I would think under the model quoted, that should be respected.

Real world justification (reaching because of lack of docs): if a module is aimed at a number of go versions, how is that specified?

[thepudds]

@kortschak, regarding your comment above about the interaction with build tags, also worth reading #31747 (comment) if you haven’t already.

[kortschak]

Thanks, @thepudds.

[gopherbot]

Change https://golang.org/cl/181840 mentions this issue: cmd/go: add more docs for go directive in go.mod file

[Helcaraxan]

Although not entirely within the scope of this issue a related piece of feedback I have seen (and heard many times more) is that the meaning of the go directive, without even speaking of its exact semantics, tends to be poorly understood by gophers that are new to modules.

The overwhelming reaction is that people believe that it will enforce a specific Go toolchain to be used. As in running your local Go 1.13 toolchain on a module with a go 1.11 directive will result in the 1.11 toolchain being downloaded and invoked instead.

The only real documentation that can be found on the web appears to be the 1.12 release notes (already referenced in the issue description) which leaves enough space for people to come to such a conclusion.

Source (one amongst many): golangci/golangci-lint#605 (comment)

[thepudds]

FWIW, I have seen a fair amount of confusion on this topic as well.

Now that the playground is running Go 1.13, here is a trivial example that illustrates one aspect of the expected behavior. Setting the Go language version to go 1.11 in the go.mod here causes a compilation error if you use signed integer shifts, which is a Go 1.13 language change:

https://play.golang.org/p/Mb7imkDHhoz

The current error is:

invalid operation: x << s (signed shift count type int, only supported as of -lang=go1.13)

In contrast, setting the Go language version in the go.mod to go 1.13 allows the use of signed integer shifts, as expected:

https://play.golang.org/p/VokbN-hHq02

That (I think?) illustrates how the go directive in a go.mod controls the version of the Go language that is used when building files within that module.

---

One particular aspect of confusion is people end up debating each other on random issues on open source projects regarding what the go value should be in the go.mod. (And it often is a "random" issue in the sense that someone is often sending a PR unrelated to modules that happens to also initially set or update the go version in the go.mod).

I think it would be helpful if there was a small bit of advice in the documentation on that piece, and I'm not sure "whatever cmd/go picks" is what most projects should be doing currently, or at least there is some nuance to the choice.

(If someone asked me, and if that someone does not yet want to use newer language features like signed integer shifts, then I'd probably personally recommend go 1.11 as a reasonable default choice currently for an open source project, given versions earlier than Go 1.11 don't even read the value. Alternatively, a recommendation could be to set it to the oldest version of Go that the open source project already knows it supports (e.g., go 1.8 or whatever). All that said, I'm not sure of the opinion of the core Go team).

[trevyn]

So just to clarify, the version set in the go directive only changes the expected language version of the source code in the module in question, and does not affect anything regarding dependencies or dependency version selection, correct?

Further, an older version of go will attempt to compile and not complain about compiling a module that specifies a newer version of go, assuming compilation succeeds?

[dmitshur]

The go directive was indeed advisory earlier on, but as of 1.21 it's no longer advisory but rather sets the minimum version of Go required to use the module. This new-to-1.21 behavior is documented at https://go.dev/doc/toolchain and at https://go.dev/ref/mod#go-mod-file-go.

So module authors should now have sufficient information needed to able to make a decision about what value to use in their modules.

Closing, since this issue was resolved in Go 1.21.