Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: verifying ...: checksum mismatch #29281

Closed
calmh opened this issue Dec 15, 2018 · 6 comments
Closed

cmd/go: verifying ...: checksum mismatch #29281

calmh opened this issue Dec 15, 2018 · 6 comments

Comments

@calmh
Copy link
Contributor

calmh commented Dec 15, 2018

go 1.11.4, linux-amd64

Today I updated our CI builders from Go 1.11.2 to 1.11.4. These are Docker images, the upgrade involved destroying the containers and starting new ones, which wiped out the build & module caches. After the upgrade one of my projects fail with this:

go: verifying github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8: checksum mismatch
  downloaded: h1:Kh7M6mzRpQ2de1rixoSQZr4BTINXFm8WDbeN5ttnwyE=
  go.sum:     h1:uZfczEBIA1FZfOQo4/JWgGnMNd/4HVsM9A+B30wtlkA=
Process exited with code 1

Since this is a hash based pseudo version I don't think the source should have changed. I still have the old zip file etc in the build cache on my workstation.

jb@kvin:~/g/p/m/c/d/g/p/p/@v $ ls -l
total 616
-rw-r--r--  1 jb  staff     105 Nov 27 09:43 list
-rw-------  1 jb  staff      78 Nov  2 06:51 v0.0.0-20170703101242-e645f4e5aaa8.info
-rw-------  1 jb  staff      36 Nov  2 06:51 v0.0.0-20170703101242-e645f4e5aaa8.mod
-rw-r--r--  1 jb  staff   73377 Nov  2 06:51 v0.0.0-20170703101242-e645f4e5aaa8.zip
...
jb@kvin:~/g/p/m/c/d/g/p/p/@v $ cat v0.0.0-20170703101242-e645f4e5aaa8.info ; echo
{"Version":"v0.0.0-20170703101242-e645f4e5aaa8","Time":"2017-07-03T10:12:42Z"}
jb@kvin:~/g/p/m/c/d/g/p/p/@v $ cat v0.0.0-20170703101242-e645f4e5aaa8.ziphash ; echo
h1:uZfczEBIA1FZfOQo4/JWgGnMNd/4HVsM9A+B30wtlkA=
jb@kvin:~/g/p/m/c/d/g/p/p/@v $ shasum v0.0.0-20170703101242-e645f4e5aaa8.zip
170651bce213135dc4e112ec0a749ae57e28f7fd  v0.0.0-20170703101242-e645f4e5aaa8.zip
jb@kvin:~/g/p/m/c/d/g/p/p/@v $ shasum v0.0.0-20170703101242-e645f4e5aaa8.mod
1e8a9dac89bb89b35fabb7c32537ccc22cbbf45e  v0.0.0-20170703101242-e645f4e5aaa8.mod

On the build server the zip file is not retained; I guess because it fails the check? So I can't compare.

root@0165ed6e0642:~/go/pkg/mod/cache/download/github.com/prometheus/procfs/@v# ls -l
total 12
-rw-r--r-- 1 root root 35 Dec 15 12:04 list
-rw------- 1 root root 78 Dec 15 12:04 v0.0.0-20170703101242-e645f4e5aaa8.info
-rw------- 1 root root 36 Dec 15 12:04 v0.0.0-20170703101242-e645f4e5aaa8.mod
root@0165ed6e0642:~/go/pkg/mod/cache/download/github.com/prometheus/procfs/@v# cat v0.0.0-20170703101242-e645f4e5aaa8.info ; echo
{"Version":"v0.0.0-20170703101242-e645f4e5aaa8","Time":"2017-07-03T10:12:42Z"}
root@0165ed6e0642:~/go/pkg/mod/cache/download/github.com/prometheus/procfs/@v# shasum v0.0.0-20170703101242-e645f4e5aaa8.mod
1e8a9dac89bb89b35fabb7c32537ccc22cbbf45e  v0.0.0-20170703101242-e645f4e5aaa8.mod

I'm not sure if this was caused by the upgrade itself, or by something external changing somehow to alter the zip hash, but I'm reporting it as it's unexpected.

@calmh
Copy link
Contributor Author

calmh commented Dec 15, 2018

The difference looks like it's caused by the upgrade. Go 1.11.2:

jb@kvin:~/t/foo $ go mod init tmp/foo
go: creating new go.mod: module tmp/foo
jb@kvin:~/t/foo $ GOPATH=~/tmp/go1112path /usr/local/go1.11.2/bin/go get github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8
go: finding github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8
go: downloading github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8
jb@kvin:~/t/foo $ cat go.sum
github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8 h1:uZfczEBIA1FZfOQo4/JWgGnMNd/4HVsM9A+B30wtlkA=
github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
jb@kvin:~/t/foo $

Go 1.11.4:

jb@kvin:~/t/bar $ go mod init tmp/bar
go: creating new go.mod: module tmp/bar
jb@kvin:~/t/bar $ GOPATH=~/tmp/go1114path /usr/local/go1.11.4/bin/go get github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8
go: finding github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8
go: downloading github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8
jb@kvin:~/t/bar $ cat go.sum
github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8 h1:Kh7M6mzRpQ2de1rixoSQZr4BTINXFm8WDbeN5ttnwyE=
github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
jb@kvin:~/t/bar $

The hash in go.sum is different. The zip files differ:

jb@kvin:~/tmp $ shasum go1112path/pkg/mod/cache/download/github.com/prometheus/procfs/\@v/v0.0.0-20170703101242-e645f4e5aaa8.zip
170651bce213135dc4e112ec0a749ae57e28f7fd  go1112path/pkg/mod/cache/download/github.com/prometheus/procfs/@v/v0.0.0-20170703101242-e645f4e5aaa8.zip

jb@kvin:~/tmp $ shasum go1114path/pkg/mod/cache/download/github.com/prometheus/procfs/\@v/v0.0.0-20170703101242-e645f4e5aaa8.zip
e3eba3d95da7ce4f011d1d06559a19961839f1d2  go1114path/pkg/mod/cache/download/github.com/prometheus/procfs/@v/v0.0.0-20170703101242-e645f4e5aaa8.zip

Looking closer at the zip files it's a bunch of odd test fixtures (symlinks) that are not included in the newer one:

e645f4e5aaa8.zip > 4
jb@kvin:~/tmp $ unzip -tv go1112path/pkg/mod/cache/download/github.com/prometheus/procfs/\@v/v0.0.0-20170703101242-e645f4e5aaa8.zip > 2
jb@kvin:~/tmp $ unzip -tv go1114path/pkg/mod/cache/download/github.com/prometheus/procfs/\@v/v0.0.0-20170703101242-jb@kvin:~/tmp $ diff -u 2 4
--- 2	2018-12-15 13:35:19.000000000 +0100
+++ 4	2018-12-15 13:35:10.000000000 +0100
@@ -1,4 +1,4 @@
-Archive:  go1112path/pkg/mod/cache/download/github.com/prometheus/procfs/@v/v0.0.0-20170703101242-e645f4e5aaa8.zip
+Archive:  go1114path/pkg/mod/cache/download/github.com/prometheus/procfs/@v/v0.0.0-20170703101242-e645f4e5aaa8.zip
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/.travis.yml   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/CONTRIBUTING.md   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/LICENSE   OK
@@ -14,23 +14,12 @@
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/doc.go   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/cmdline   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/comm   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/exe   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/fd/0   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/fd/1   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/fd/10   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/fd/2   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/fd/3   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/io   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/limits   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/mountstats   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26231/stat   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/cmdline   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/comm   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/fd/0   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/fd/1   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/fd/2   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/fd/3   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/fd/4   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/limits   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/26232/stat   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/584/stat   OK
@@ -42,7 +31,6 @@
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/net/ip_vs   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/net/ip_vs_stats   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/net/xfrm_stat   OK
-    testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/self   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/stat   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/symlinktargets/README   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/fixtures/symlinktargets/abc   OK
@@ -79,4 +67,4 @@
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/xfs/parse.go   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/xfs/parse_test.go   OK
     testing: github.com/prometheus/procfs@v0.0.0-20170703101242-e645f4e5aaa8/xfs/xfs.go   OK
-No errors detected in compressed data of go1112path/pkg/mod/cache/download/github.com/prometheus/procfs/@v/v0.0.0-20170703101242-e645f4e5aaa8.zip.
+No errors detected in compressed data of go1114path/pkg/mod/cache/download/github.com/prometheus/procfs/@v/v0.0.0-20170703101242-e645f4e5aaa8.zip.
[1]
jb@kvin:~/tmp $

I guess this is probably good and intentional, but it's a surprising and annoying failure for a minor update...

@ALTree
Copy link
Member

ALTree commented Dec 15, 2018

Looking closer at the zip files it's a bunch of odd test fixtures (symlinks) that are not included in the newer one

Yeah. This is #27093 and the fix was backported to 1.11.4 (#29191).

The owner of the cmd/go modules code is aware of this (see #29278 (comment)), so it's safe to say that this was done on purpose, even if it's true this can be a little annoying.

I'd close here as WAI.

@calmh
Copy link
Contributor Author

calmh commented Dec 15, 2018

Right... I didn't find that issue in a search for whatever reason.

@calmh calmh closed this as completed Dec 15, 2018
@thepudds
Copy link
Contributor

thepudds commented Dec 15, 2018

Note: if you have a copy of a module with symlinks in your module cache, you likely will need to run go clean -modcache in order to get 1.11.4 to generate the new (corrected) checksum. (e.g., see #29282)

dhewg added a commit to dhewg/packages that referenced this issue Jan 3, 2019
This fixes the exact same build error reported at [0].
Upstream fixed it too, see [1], but I decided to use the minimal patch
until a new version is released instead.

[0] golang/go#29281
[1] prometheus/statsd_exporter#171

Signed-off-by: Andre Heider <a.heider@gmail.com>
dhewg added a commit to dhewg/packages that referenced this issue Jan 4, 2019
This fixes the exact same build error reported at [0].
Upstream fixed it too, see [1], but I decided to use the minimal patch
until a new version is released instead.

[0] golang/go#29281
[1] prometheus/statsd_exporter#171

Signed-off-by: Andre Heider <a.heider@gmail.com>
@Koshmaar
Copy link

Koshmaar commented Jan 9, 2019

I had similiar problem inside Dockerfile. I solved it using RUN cd project; rm go.sum; go clean -modcache; make linux - it was enough to do it once, then you can remove two middle commands. And obviously you need golang 1.11.4 (or latest).

ClaymorePT pushed a commit to ClaymorePT/packages that referenced this issue Feb 12, 2019
This fixes the exact same build error reported at [0].
Upstream fixed it too, see [1], but I decided to use the minimal patch
until a new version is released instead.

[0] golang/go#29281
[1] prometheus/statsd_exporter#171

Signed-off-by: Andre Heider <a.heider@gmail.com>
@dkushner
Copy link

@Koshmaar Any idea what exactly is breaking here? I have run our build in golang:1.11.5-alpine in a dead sterile environment where Docker should have no cached images to pull from and still encountered this issue. What exactly is going on that I would need to clear the modcache in this scenario?

thiagoricciardi pushed a commit to thiagoricciardi/packages that referenced this issue Apr 3, 2019
This fixes the exact same build error reported at [0].
Upstream fixed it too, see [1], but I decided to use the minimal patch
until a new version is released instead.

[0] golang/go#29281
[1] prometheus/statsd_exporter#171

Signed-off-by: Andre Heider <a.heider@gmail.com>
foxcpp added a commit to foxcpp/go-jmap that referenced this issue May 18, 2019
@golang golang locked and limited conversation to collaborators Feb 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants