Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go install downloads too much #45004

Open
aarondl opened this issue Mar 14, 2021 · 4 comments
Open

cmd/go: go install downloads too much #45004

aarondl opened this issue Mar 14, 2021 · 4 comments
Labels
modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. ToolSpeed
Milestone
Backlog

Comments

@aarondl
Copy link

aarondl commented Mar 14, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/aaron/.cache/go-build"
GOENV="/home/aaron/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/aaron/TESTDOWNLOAD/GOPATH/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/aaron/TESTDOWNLOAD/GOPATH"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2663290531=/tmp/go-build -gno-record-gcc-switches"

What did you do?

mkdir -p ~/TESTDOWNLOAD/GOPATH
cd ~/TESTDOWNLOAD
export GOPATH=$PWD/GOPATH

go install github.com/volatiletech/sqlboiler/v4@latest

What did you expect to see?

I expected to see only sqlboiler/v4 downloaded installed.

What did you see instead?

I additionally saw it downloading source for versions I was not interested in at all nor appear to be part of the dependency tree. Note the extraneous downloads of the source for v1.0.0 and v3.7.1+incompatible.

# get https://proxy.golang.org/github.com/@v/list
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/list
# get https://proxy.golang.org/github.com/volatiletech/@v/list
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/list
# get https://proxy.golang.org/github.com/volatiletech/@v/list: 410 Gone (0.113s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/list: 200 OK (0.118s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.info
# get https://proxy.golang.org/github.com/@v/list: 410 Gone (0.119s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/list: 200 OK (0.125s)
go: downloading github.com/volatiletech/sqlboiler v1.0.0
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v1.0.0.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.info: 200 OK (0.009s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.mod
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v1.0.0.zip: 200 OK (0.009s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.mod: 200 OK (0.025s)
# get https://proxy.golang.org/sumdb/sum.golang.org/supported
# get https://proxy.golang.org/sumdb/sum.golang.org/supported: 410 Gone (0.008s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler/v4@v4.5.0
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v1.0.0
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler/v4@v4.5.0: 200 OK (0.095s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v1.0.0: 200 OK (0.095s)
# get https://sum.golang.org/tile/8/0/x012/871.p/183
# get https://sum.golang.org/tile/8/2/000.p/50
# get https://sum.golang.org/tile/8/1/050.p/71
# get https://sum.golang.org/tile/8/2/000.p/50: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/1/050.p/71: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/x012/871.p/183: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/430
# get https://sum.golang.org/tile/8/1/001
# get https://sum.golang.org/tile/8/1/001: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/430: 200 OK (0.010s)
go: downloading github.com/volatiletech/sqlboiler/v4 v4.5.0
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.zip: 200 OK (0.011s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.info
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.info: 200 OK (0.046s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.mod
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.mod: 200 OK (0.008s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v3.7.1+incompatible
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v3.7.1+incompatible: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/0/x004/453
# get https://sum.golang.org/tile/8/1/017
# get https://sum.golang.org/tile/8/0/x004/453: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/1/017: 200 OK (0.009s)
go: downloading github.com/volatiletech/sqlboiler v3.7.1+incompatible
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.zip: 200 OK (0.011s)

Inside the project in question, I tried some queries to see if maybe there was something pulling in the old versions:

$ go mod graph | grep sqlboiler
github.com/volatiletech/sqlboiler/v4 github.com/DATA-DOG/go-sqlmock@v1.4.1
github.com/volatiletech/sqlboiler/v4 github.com/davecgh/go-spew@v1.1.1
github.com/volatiletech/sqlboiler/v4 github.com/denisenkom/go-mssqldb@v0.0.0-20200206145737-bbfc9a55622e
github.com/volatiletech/sqlboiler/v4 github.com/ericlagergren/decimal@v0.0.0-20181231230500-73749d4874d5
github.com/volatiletech/sqlboiler/v4 github.com/friendsofgo/errors@v0.9.2
github.com/volatiletech/sqlboiler/v4 github.com/go-sql-driver/mysql@v1.5.0
github.com/volatiletech/sqlboiler/v4 github.com/kat-co/vala@v0.0.0-20170210184112-42e1d8b61f12
github.com/volatiletech/sqlboiler/v4 github.com/lib/pq@v1.2.1-0.20191011153232-f91d3411e481
github.com/volatiletech/sqlboiler/v4 github.com/spf13/cast@v1.3.1
github.com/volatiletech/sqlboiler/v4 github.com/spf13/cobra@v1.0.0
github.com/volatiletech/sqlboiler/v4 github.com/spf13/viper@v1.6.3
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/null/v8@v8.1.2
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/randomize@v0.0.1
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/strmangle@v0.0.1
github.com/volatiletech/sqlboiler/v4 gopkg.in/yaml.v2@v2.2.4

$ go list -m all | grep sqlboiler
github.com/volatiletech/sqlboiler/v4

$ go list all | grep sqlboiler
github.com/volatiletech/sqlboiler/v4
github.com/volatiletech/sqlboiler/v4/boil
github.com/volatiletech/sqlboiler/v4/boilingcore
github.com/volatiletech/sqlboiler/v4/drivers
github.com/volatiletech/sqlboiler/v4/drivers/mocks
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mssql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mssql/driver
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mysql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mysql/driver
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-psql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-psql/driver
github.com/volatiletech/sqlboiler/v4/importers
github.com/volatiletech/sqlboiler/v4/models
github.com/volatiletech/sqlboiler/v4/queries
github.com/volatiletech/sqlboiler/v4/queries/qm
github.com/volatiletech/sqlboiler/v4/queries/qmhelper
github.com/volatiletech/sqlboiler/v4/templatebin
github.com/volatiletech/sqlboiler/v4/types
github.com/volatiletech/sqlboiler/v4/types/pgeo

Also as a completely separate nit trying go list all without changes in the repo produces the error:

missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
	go get -t gopkg.in/yaml.v2@v2.2.4

Running this command changes nothing and the error continues, the fix is in fact running go mod download which adds many lines to the go.sum file. Perhaps I should add these and commit them as the maintainer of the package.
@zikaeroh
Copy link
Contributor

RE: your last point, apparently it's expected that go mod tidy and go mod download do not result in the same go.sum anymore. See #43994 (and the many linked issues and workarounds people have been applying in other repos...).

@cherrymui cherrymui added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 15, 2021
@cherrymui cherrymui added this to the Backlog milestone Mar 15, 2021
@cherrymui
Copy link
Member

cc @bcmills @jayconrod @matloob

@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2021

Those source downloads are because the go command is looking for a package named at ./v4 within module github.com/volatiletech/sqlboiler. For go get that's maybe not such a big deal because the main module will in general already require the module at the correct version, but for go install I agree that it seems excessive.

@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2021

Also as a completely separate nit trying go list all without changes in the repo produces the error:

missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
	go get -t gopkg.in/yaml.v2@v2.2.4

Running this command changes nothing and the error continues

I'm not able to reproduce that behavior:

sqlboiler$ go list all >/dev/null
missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
        go get -t gopkg.in/yaml.v2@v2.2.4

sqlboiler$ go get -t gopkg.in/yaml.v2@v2.2.4

sqlboiler$ go list all >/dev/null

sqlboiler$

However, that sounds like it could be #44749. The fix for that (#44812) just barely missed 1.16.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. ToolSpeed
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
4 participants
@aarondl @bcmills @cherrymui @zikaeroh