cmd/go: [modules + integration] go mod pack, pack sources as module files

[nim-nim]

This report is part of a series, filled at the request of @mdempsky, focused at making Go modules integrator-friendly.

Please do not close or mark it as duplicate before making sure you’ve read and understood the general context. A lot of work went into identifying problems points precisely.

Needed feature

Go needs an official go mod pack command that processes a set of unpacked Go modules and generates the corresponding packed module files for reuse (as specified in goproxy.)

This is different from issue #27858, because issue #27858 wants to put the generated files in the module cache, mixing them with modules of other provenance, and forbidding result reuse by anyone but the current user.

Constrains

• the feature should also be exposed as a function in the official Go API
• the input set may be defined as one or several lists of of go.mod filesystem paths (as produced by go mod discover in issue cmd/go: [modules + integration] go mod discover, discover a set of unpacked modules within a directory #31299), one or several directory paths (similar to the directory paths defined in issue cmd/go: [modules + integration] go mod discover, discover a set of unpacked modules within a directory #31299), or a mix of both
• the destination should be any binary directory path the user specifies
• for compatibility with existing tooling, a separate optional prefix flag should allow pre-pending a path to the destination:
  • ie pretend working on destination, actually work on filepath.Join(prefix, destination)
  • that is typically used to prepare deployment to canonical_path, using a /prefix/canonical_path staging directory
• to allow integration with CI/CD software, the command should optionally output a list of the files it created, in machine-readable format, to a user-specified result file
  • the result file path is not affected by destination
  • the result file lists paths without prefix, since the command is pretending to write directly into destination
  • this is similar to the behaviour of go mod build (cmd/go: [modules + integration] go mod build, build all the binaries provided by a module #31323) and should use the same conventions
• the command should use versioning info present in the info files discovered next to the corresponding mod files
• the command should also take user-provided versioning info as input, either to fill in blanks when info files are not present, or to override them (both strategies could arguably be valid)
• the command should tidy the mod files by default, removing unneeded requires (but see also cmd/go: [modules + integration] tidy a specific module descriptor #31318)
• the command should work in secure no-internet-download mode. In that mode it should probably restricts its tidying to direct dependencies and the dependencies available in configured goproxy sources (cmd/go: [modules + integration] use several goproxy sources simultaneously #31304)
• the command may also generate/update the corresponding list index files (but see also cmd/go: [modules + integration] go mod index, reindex a goproxy directory #31303 that is more general and is needed anyway)

Motivation

Creating and managing a baseline of third-party code in a go module world requires the ability to generate the go module files that will serve as baseline blocks in separate CI/CD runs.

[agnivade]

@bcmills @jayconrod

[gopherbot]

Change https://golang.org/cl/202042 mentions this issue: zip: add package for creating and extracting module zip files

[seankhliao]

The api is https://pkg.go.dev/golang.org/x/mod/zip