Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go install downloads too much #45004

Open
aarondl opened this issue Mar 14, 2021 · 4 comments
Open

cmd/go: go install downloads too much #45004

aarondl opened this issue Mar 14, 2021 · 4 comments
Labels
modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. ToolSpeed
Milestone

Comments

@aarondl
Copy link

aarondl commented Mar 14, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/aaron/.cache/go-build"
GOENV="/home/aaron/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/aaron/TESTDOWNLOAD/GOPATH/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/aaron/TESTDOWNLOAD/GOPATH"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2663290531=/tmp/go-build -gno-record-gcc-switches"

What did you do?

mkdir -p ~/TESTDOWNLOAD/GOPATH
cd ~/TESTDOWNLOAD
export GOPATH=$PWD/GOPATH

go install github.com/volatiletech/sqlboiler/v4@latest

What did you expect to see?

I expected to see only sqlboiler/v4 downloaded installed.

What did you see instead?

I additionally saw it downloading source for versions I was not interested in at all nor appear to be part of the dependency tree. Note the extraneous downloads of the source for v1.0.0 and v3.7.1+incompatible.

# get https://proxy.golang.org/github.com/@v/list
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/list
# get https://proxy.golang.org/github.com/volatiletech/@v/list
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/list
# get https://proxy.golang.org/github.com/volatiletech/@v/list: 410 Gone (0.113s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/list: 200 OK (0.118s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.info
# get https://proxy.golang.org/github.com/@v/list: 410 Gone (0.119s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/list: 200 OK (0.125s)
go: downloading github.com/volatiletech/sqlboiler v1.0.0
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v1.0.0.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.info: 200 OK (0.009s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.mod
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v1.0.0.zip: 200 OK (0.009s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.mod: 200 OK (0.025s)
# get https://proxy.golang.org/sumdb/sum.golang.org/supported
# get https://proxy.golang.org/sumdb/sum.golang.org/supported: 410 Gone (0.008s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler/v4@v4.5.0
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v1.0.0
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler/v4@v4.5.0: 200 OK (0.095s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v1.0.0: 200 OK (0.095s)
# get https://sum.golang.org/tile/8/0/x012/871.p/183
# get https://sum.golang.org/tile/8/2/000.p/50
# get https://sum.golang.org/tile/8/1/050.p/71
# get https://sum.golang.org/tile/8/2/000.p/50: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/1/050.p/71: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/x012/871.p/183: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/430
# get https://sum.golang.org/tile/8/1/001
# get https://sum.golang.org/tile/8/1/001: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/430: 200 OK (0.010s)
go: downloading github.com/volatiletech/sqlboiler/v4 v4.5.0
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.zip: 200 OK (0.011s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.info
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.info: 200 OK (0.046s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.mod
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.mod: 200 OK (0.008s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v3.7.1+incompatible
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v3.7.1+incompatible: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/0/x004/453
# get https://sum.golang.org/tile/8/1/017
# get https://sum.golang.org/tile/8/0/x004/453: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/1/017: 200 OK (0.009s)
go: downloading github.com/volatiletech/sqlboiler v3.7.1+incompatible
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.zip: 200 OK (0.011s)

Inside the project in question, I tried some queries to see if maybe there was something pulling in the old versions:

$ go mod graph | grep sqlboiler
github.com/volatiletech/sqlboiler/v4 github.com/DATA-DOG/go-sqlmock@v1.4.1
github.com/volatiletech/sqlboiler/v4 github.com/davecgh/go-spew@v1.1.1
github.com/volatiletech/sqlboiler/v4 github.com/denisenkom/go-mssqldb@v0.0.0-20200206145737-bbfc9a55622e
github.com/volatiletech/sqlboiler/v4 github.com/ericlagergren/decimal@v0.0.0-20181231230500-73749d4874d5
github.com/volatiletech/sqlboiler/v4 github.com/friendsofgo/errors@v0.9.2
github.com/volatiletech/sqlboiler/v4 github.com/go-sql-driver/mysql@v1.5.0
github.com/volatiletech/sqlboiler/v4 github.com/kat-co/vala@v0.0.0-20170210184112-42e1d8b61f12
github.com/volatiletech/sqlboiler/v4 github.com/lib/pq@v1.2.1-0.20191011153232-f91d3411e481
github.com/volatiletech/sqlboiler/v4 github.com/spf13/cast@v1.3.1
github.com/volatiletech/sqlboiler/v4 github.com/spf13/cobra@v1.0.0
github.com/volatiletech/sqlboiler/v4 github.com/spf13/viper@v1.6.3
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/null/v8@v8.1.2
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/randomize@v0.0.1
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/strmangle@v0.0.1
github.com/volatiletech/sqlboiler/v4 gopkg.in/yaml.v2@v2.2.4

$ go list -m all | grep sqlboiler
github.com/volatiletech/sqlboiler/v4

$ go list all | grep sqlboiler
github.com/volatiletech/sqlboiler/v4
github.com/volatiletech/sqlboiler/v4/boil
github.com/volatiletech/sqlboiler/v4/boilingcore
github.com/volatiletech/sqlboiler/v4/drivers
github.com/volatiletech/sqlboiler/v4/drivers/mocks
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mssql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mssql/driver
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mysql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mysql/driver
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-psql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-psql/driver
github.com/volatiletech/sqlboiler/v4/importers
github.com/volatiletech/sqlboiler/v4/models
github.com/volatiletech/sqlboiler/v4/queries
github.com/volatiletech/sqlboiler/v4/queries/qm
github.com/volatiletech/sqlboiler/v4/queries/qmhelper
github.com/volatiletech/sqlboiler/v4/templatebin
github.com/volatiletech/sqlboiler/v4/types
github.com/volatiletech/sqlboiler/v4/types/pgeo

Also as a completely separate nit trying go list all without changes in the repo produces the error:

missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
	go get -t gopkg.in/yaml.v2@v2.2.4

Running this command changes nothing and the error continues, the fix is in fact running go mod download which adds many lines to the go.sum file. Perhaps I should add these and commit them as the maintainer of the package.

@zikaeroh
Copy link
Contributor

RE: your last point, apparently it's expected that go mod tidy and go mod download do not result in the same go.sum anymore. See #43994 (and the many linked issues and workarounds people have been applying in other repos...).

@cherrymui cherrymui added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 15, 2021
@cherrymui cherrymui added this to the Backlog milestone Mar 15, 2021
@cherrymui
Copy link
Member

cc @bcmills @jayconrod @matloob

@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2021

Those source downloads are because the go command is looking for a package named at ./v4 within module github.com/volatiletech/sqlboiler. For go get that's maybe not such a big deal because the main module will in general already require the module at the correct version, but for go install I agree that it seems excessive.

@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2021

Also as a completely separate nit trying go list all without changes in the repo produces the error:

missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
	go get -t gopkg.in/yaml.v2@v2.2.4

Running this command changes nothing and the error continues

I'm not able to reproduce that behavior:

sqlboiler$ go list all >/dev/null
missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
        go get -t gopkg.in/yaml.v2@v2.2.4

sqlboiler$ go get -t gopkg.in/yaml.v2@v2.2.4

sqlboiler$ go list all >/dev/null

sqlboiler$

However, that sounds like it could be #44749. The fix for that (#44812) just barely missed 1.16.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. ToolSpeed
Projects
None yet
Development

No branches or pull requests

4 participants