Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: GO111MODULE=on and unset GOPATH generate incompatible module checksums #29664

Closed
jayschwa opened this issue Jan 10, 2019 · 7 comments
Closed

cmd/go: GO111MODULE=on and unset GOPATH generate incompatible module checksums #29664

jayschwa opened this issue Jan 10, 2019 · 7 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@jayschwa
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.11.4 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jay/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jay/spell"
GOPROXY=""
GORACE=""
GOROOT="/snap/go/3095"
GOTMPDIR=""
GOTOOLDIR="/snap/go/3095/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build944689084=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ rm go.sum
$ GO111MODULE=on go mod tidy
$ (unset GOPATH && go mod tidy)
go: verifying github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767: checksum mismatch
	downloaded: h1:l6MPrA7HbTIeBptIanRxdS9gn7DhOjBUJr6gxO0ssKc=
	go.sum:     h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=

$ rm go.sum
$ (unset GOPATH && go mod tidy)
$ GO111MODULE=on go mod tidy
go: verifying github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767: checksum mismatch
	downloaded: h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=
	go.sum:     h1:l6MPrA7HbTIeBptIanRxdS9gn7DhOjBUJr6gxO0ssKc=

What did you expect to see?

I expected the same (or at least compatible) go.sum to be generated
@julieqiu julieqiu added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 10, 2019
@julieqiu
Copy link
Member

/cc @bcmills

@bcmills
Copy link
Contributor

bcmills commented Jan 10, 2019

Dup of #29278. Run go clean -modcache and clear out the go.sum entries for any modules whose repos contain symlinks.

@bcmills bcmills closed this as completed Jan 10, 2019
@jayschwa
Copy link
Contributor Author

@bcmills, this is not a duplicate. The problem still occurs after go clean -modcache.

@bcmills
Copy link
Contributor

bcmills commented Jan 10, 2019

Yes, it is a duplicate. Read the whole reply.

and clear out the go.sum entries for any modules whose repos contain symlinks.

~/src/github.com/docker/docker$ find ./* -type l
./integration-cli/fixtures/https/server-key.pem
./integration-cli/fixtures/https/client-cert.pem
./integration-cli/fixtures/https/server-cert.pem
./integration-cli/fixtures/https/client-key.pem
./integration-cli/fixtures/https/ca.pem

@jayschwa
Copy link
Contributor Author

Read the whole bug report. The first step of my example is rm go.sum.

@bcmills
Copy link
Contributor

bcmills commented Jan 10, 2019

Yes, and you ran unset GOPATH && go mod tidy. Did you also unset GOPATH when you ran go clean -modcache?

Here's what I get, starting from a clean GOPATH:

$ go1.11.4 mod init golang.org/issue/scratch
go: creating new go.mod: module golang.org/issue/scratch

$ cat >main.go <<EOF
package main
import _ "github.com/docker/docker/client"
func main() {}
EOF

$ go1.11.4 get -m github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767
go: finding github.com/docker/docker v0.7.3-0.20181128160009-852542b39767

$ go1.11.4 mod tidy
go: downloading github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: finding github.com/docker/go-connections/tlsconfig latest
go: finding github.com/docker/go-connections/nat latest
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/docker/distribution/reference latest
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: downloading github.com/docker/go-units v0.3.3
go: finding github.com/docker/go-connections v0.4.0
go: downloading github.com/pkg/errors v0.8.1
go: finding gotest.tools/env latest
go: finding gotest.tools/assert/cmp latest
go: finding gotest.tools/skip latest
go: finding gotest.tools/assert latest
go: finding github.com/sirupsen/logrus v1.3.0
go: finding github.com/gorilla/mux v1.6.2
go: downloading github.com/docker/go-connections v0.4.0
go: finding github.com/docker/go-connections/sockets latest
go: finding google.golang.org/grpc/codes latest
go: finding github.com/opencontainers/image-spec/specs-go latest
go: finding github.com/Azure/go-ansiterm/winterm latest
go: downloading github.com/sirupsen/logrus v1.3.0
go: downloading github.com/gorilla/mux v1.6.2
go: finding google.golang.org/grpc v1.17.0
go: finding gotest.tools v2.2.0+incompatible
go: downloading gotest.tools v2.2.0+incompatible
go: finding github.com/Azure/go-ansiterm latest
go: downloading github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding github.com/google/go-cmp/cmp latest
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding github.com/google/go-cmp/cmp/cmpopts latest
go: finding github.com/gogo/protobuf/proto latest
go: finding github.com/Nvveen/Gotty latest
go: finding github.com/google/go-cmp v0.2.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: downloading google.golang.org/grpc v1.17.0
go: downloading github.com/google/go-cmp v0.2.0
go: finding golang.org/x/time/rate latest
go: finding golang.org/x/sys/unix latest
go: finding github.com/docker/distribution v2.7.0+incompatible
go: downloading github.com/docker/distribution v2.7.0+incompatible
go: finding github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/time latest
go: downloading golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: downloading github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/golang/protobuf v1.2.0
go: finding github.com/golang/mock v1.1.1
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding google.golang.org/appengine v1.1.0
go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858
go: finding cloud.google.com/go v0.26.0
go: finding golang.org/x/text v0.3.0
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding github.com/client9/misspell v0.3.4
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52
go: finding github.com/kisielk/gotool v1.0.0
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding github.com/stretchr/objx v0.1.1
go: downloading golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: downloading github.com/stretchr/testify v1.2.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: downloading golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: downloading google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: downloading github.com/golang/protobuf v1.2.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/Microsoft/go-winio v0.4.11
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: downloading golang.org/x/text v0.3.0
go: downloading github.com/Microsoft/go-winio v0.4.11
go: downloading github.com/gorilla/context v1.1.1

$ GOPATH=$(mktemp -d) go1.11.4 mod tidy
go: finding github.com/Microsoft/go-winio v0.4.11
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/docker/distribution v2.7.0+incompatible
go: finding github.com/docker/go-connections v0.4.0
go: finding github.com/google/go-cmp v0.2.0
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding gotest.tools v2.2.0+incompatible
go: finding github.com/gorilla/mux v1.6.2
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: finding github.com/sirupsen/logrus v1.3.0
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: finding google.golang.org/grpc v1.17.0
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/stretchr/objx v0.1.1
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding github.com/golang/protobuf v1.2.0
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding google.golang.org/appengine v1.1.0
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858
go: finding golang.org/x/text v0.3.0
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding github.com/client9/misspell v0.3.4
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding github.com/kisielk/gotool v1.0.0
go: finding github.com/golang/mock v1.1.1
go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding cloud.google.com/go v0.26.0
go: downloading github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: extracting github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: downloading google.golang.org/grpc v1.17.0
go: downloading gotest.tools v2.2.0+incompatible
go: downloading github.com/docker/distribution v2.7.0+incompatible
go: downloading github.com/docker/go-connections v0.4.0
go: downloading github.com/pkg/errors v0.8.1
go: downloading github.com/gorilla/mux v1.6.2
go: extracting github.com/opencontainers/go-digest v1.0.0-rc1
go: extracting github.com/docker/go-connections v0.4.0
go: extracting github.com/pkg/errors v0.8.1
go: downloading github.com/Microsoft/go-winio v0.4.11
go: downloading golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: extracting github.com/gorilla/mux v1.6.2
go: extracting gotest.tools v2.2.0+incompatible
go: downloading github.com/gorilla/context v1.1.1
go: downloading github.com/docker/go-units v0.3.3
go: downloading github.com/sirupsen/logrus v1.3.0
go: downloading github.com/google/go-cmp v0.2.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: extracting github.com/docker/go-units v0.3.3
go: extracting github.com/gorilla/context v1.1.1
go: downloading github.com/stretchr/testify v1.2.2
go: extracting github.com/Microsoft/go-winio v0.4.11
go: downloading github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: downloading github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: extracting github.com/google/go-cmp v0.2.0
go: extracting github.com/sirupsen/logrus v1.3.0
go: downloading golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: extracting github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: downloading github.com/gogo/protobuf v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: extracting github.com/stretchr/testify v1.2.2
go: extracting github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: downloading github.com/davecgh/go-spew v1.1.1
go: extracting github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/pmezard/go-difflib v1.0.0
go: extracting github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: extracting github.com/davecgh/go-spew v1.1.1
go: downloading golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: extracting google.golang.org/grpc v1.17.0
go: extracting github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: extracting golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: downloading google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: extracting github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading github.com/golang/protobuf v1.2.0
go: extracting github.com/docker/distribution v2.7.0+incompatible
go: extracting golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: extracting golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: downloading golang.org/x/text v0.3.0
go: extracting github.com/golang/protobuf v1.2.0
go: downloading golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: extracting golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: extracting golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: extracting github.com/gogo/protobuf v1.2.0
go: extracting google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: extracting golang.org/x/text v0.3.0

$ cat go.sum | grep docker/docker
github.com/docker/docker v0.7.3-0.20181128160009-852542b39767 h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=
github.com/docker/docker v0.7.3-0.20181128160009-852542b39767/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=


$ GOPATH=$(mktemp -d) go1.11.2 mod tidy
go: finding github.com/sirupsen/logrus v1.3.0
go: finding github.com/gorilla/mux v1.6.2
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: finding github.com/docker/distribution v2.7.0+incompatible
go: finding github.com/google/go-cmp v0.2.0
go: finding github.com/Microsoft/go-winio v0.4.11
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding gotest.tools v2.2.0+incompatible
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/docker/go-connections v0.4.0
go: finding github.com/stretchr/testify v1.2.2
go: finding golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding google.golang.org/grpc v1.17.0
go: finding github.com/stretchr/objx v0.1.1
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858
go: finding google.golang.org/appengine v1.1.0
go: finding github.com/client9/misspell v0.3.4
go: finding cloud.google.com/go v0.26.0
go: finding github.com/golang/mock v1.1.1
go: finding github.com/golang/protobuf v1.2.0
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding github.com/kisielk/gotool v1.0.0
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52
go: finding golang.org/x/text v0.3.0
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: verifying github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767: checksum mismatch
        downloaded: h1:l6MPrA7HbTIeBptIanRxdS9gn7DhOjBUJr6gxO0ssKc=
        go.sum:     h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=

The error in your snippet is telling you exactly what is wrong: the module cache in your default GOPATH (that is, $USER/go/pkg/mod) has an incorrect (pre-1.11.4) copy of the module, while the cache in your actual GOPATH ($GOPATH/pkg/mod) has the correct one.

@jayschwa
Copy link
Contributor Author

Did you also unset GOPATH when you ran go clean -modcache?

No, that explains it. Sorry for the bother (and salt).

pwaller added a commit to pwaller/docker-show-context that referenced this issue Jan 20, 2019
@pwaller
Fix docker go.sum for golang/go#29664 (fixes #13)
60e4223
@pwaller pwaller mentioned this issue Jan 20, 2019
Closed
@mbana mbana mentioned this issue Jan 20, 2019
Closed
jirfag added a commit to golangci/golangci-lint that referenced this issue Jan 21, 2019
@jirfag
Fix #344: fix go-misc checksum on go1.11.4
59a533c 
See golang/go#29664 and golang/go#29278
mrnugget added a commit to dcos/dcos-go that referenced this issue Jan 23, 2019
@mrnugget
Regenerate go.sum with cleared cache
4b7162d 
The problem that caused our Jenkins jobs to fail due to a "checksum
mismatch" is exactly this one: golang/go#29664

In order to fix it, I removed the `go.sum` file and recreated it, which
lead to the two checksums being updated.
@mrnugget mrnugget mentioned this issue Jan 23, 2019
Merged
@golang golang locked and limited conversation to collaborators Jan 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jayschwa @bcmills @julieqiu @gopherbot