Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: duplicate lines for module in go.sum #28456

Closed
myitcv opened this issue Oct 29, 2018 · 10 comments
Closed

cmd/go: duplicate lines for module in go.sum #28456

myitcv opened this issue Oct 29, 2018 · 10 comments
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@myitcv
Copy link
Member

myitcv commented Oct 29, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.11.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/tmp/tmp.UI0ocKgDAb"
GOPROXY=""
GORACE=""
GOROOT="/home/myitcv/gos"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/tmp.cftvdIn4ct/hello/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build473745155=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I don't yet have a full repro, but the following PR shows the effect:

myitcv/gobin#19

Reporting here in case others see the issue and can add any context/detail.

What did you expect to see?

No duplicate lines in go.sum

What did you see instead?

$ diff -wu <(cat go.sum | sort) <(cat go.sum | sort | uniq)
--- /dev/fd/63  2018-10-29 11:28:53.817890106 +0000
+++ /dev/fd/62  2018-10-29 11:28:53.813890308 +0000
@@ -18,8 +18,6 @@
 github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
 github.com/onsi/gomega v1.4.1/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
 github.com/rogpeppe/go-internal v1.0.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
-github.com/rogpeppe/go-internal v1.0.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
-github.com/rogpeppe/go-internal v1.0.0 h1:o4VLZ5jqHE+HahLT6drNtSGTrrUA3wPBmtpgqtdbClo=
 github.com/rogpeppe/go-internal v1.0.0 h1:o4VLZ5jqHE+HahLT6drNtSGTrrUA3wPBmtpgqtdbClo=
 github.com/russross/blackfriday v1.5.1/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
 github.com/sclevine/agouti v3.0.0+incompatible/go.mod h1:b4WX9W9L1sfQKXeJf1mUTLZKJ48R1S7H23Ji7oFO5Bw=
@myitcv myitcv added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go modules labels Oct 29, 2018
@myitcv myitcv added this to the Go1.12 milestone Oct 29, 2018
@bcmills
Copy link
Contributor

bcmills commented Nov 16, 2018

This may go away on its own if we add go.sum merging logic for #28802.

@bcmills bcmills modified the milestones: Go1.12, Go1.13 Nov 16, 2018
@neuromage neuromage mentioned this issue Jan 8, 2019
Merged
@tv42
Copy link

tv42 commented Feb 12, 2019

For what it's worth, I ended up with duplicate go.sum lines via lazy conflict resolution, and was surprised that go mod tidy did not clean it up.

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@myitcv
Copy link
Member Author

myitcv commented Jul 12, 2019

Just to confirm this remains an issue in 1.13beta1:

env GOPROXY=proxy.golang.org

go mod tidy
cmp go.sum go.sum.golden
cp go.sum.dup go.sum
go mod tidy
cmp go.sum go.sum.golden

-- go.mod --
module mod
-- go.sum.dup --
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0 h1:0vLT13EuvQ0hNvakwLuFZ/jYrLp5F3kcWHXdRggjCE8=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
-- go.sum.golden --
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0 h1:0vLT13EuvQ0hNvakwLuFZ/jYrLp5F3kcWHXdRggjCE8=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
-- main.go --
package main

import _ "github.com/rogpeppe/go-internal/testscript"

func main() {}

Gives:

> env GOPROXY=proxy.golang.org
> go mod tidy
[stderr]
go: finding github.com/rogpeppe/go-internal v1.3.0
go: downloading github.com/rogpeppe/go-internal v1.3.0
go: extracting github.com/rogpeppe/go-internal v1.3.0
go: downloading gopkg.in/errgo.v2 v2.1.0
go: extracting gopkg.in/errgo.v2 v2.1.0
go: downloading gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: extracting gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: downloading github.com/kr/pretty v0.1.0
go: extracting github.com/kr/pretty v0.1.0
go: downloading github.com/kr/text v0.1.0
go: extracting github.com/kr/text v0.1.0

> cmp go.sum go.sum.golden
> cp go.sum.dup go.sum
> go mod tidy
> cmp go.sum go.sum.golden
[diff -go.sum +go.sum.golden]
 github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
-github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
-github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/errgo.v2 v2.1.0 h1:0vLT13EuvQ0hNvakwLuFZ/jYrLp5F3kcWHXdRggjCE8=
 gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=

FAIL: /tmp/testscript286011127/0/script.txt:7: go.sum and go.sum.golden differ
error running go1.13_mod_tidy_dup_bug.txt in /tmp/testscript286011127/0

@bcmills bcmills mentioned this issue Aug 12, 2019
Closed
@bcmills
Copy link
Contributor

bcmills commented Aug 12, 2019

In #33603, @kevinburkemeter mentions that he got into this state via a bad rebase.

@aarondl aarondl mentioned this issue Sep 2, 2019
Closed
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@xueweiz xueweiz mentioned this issue Nov 26, 2019
Closed
@honnix
Copy link

honnix commented Jan 3, 2020 •
edited

The problem I experienced is a bit different.

+github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
 github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=

The file tree checksum is not populated in go.sum on my computer (macOS) but added by TravisCI (Linux). I use the same version of Go for both.

Further more, if I manually add the file tree checksum locally to go.sum and run go mod tidy afterwards, it is not removed. In other words, go mod tidy is happy with either having it or not.

Updated:

I managed to find a difference between my local build and CI. There is an additional go get ... executed in CI and that results change of go.sum. This could boil down to #34506, although it is still weird go mod tidy does not clean up accordingly.

@jayconrod
Copy link
Contributor

@honnix That sounds different than this issue. Would you please open a new one? go mod tidy should end up with a canonical set of hashes.

@honnix
Copy link

honnix commented Jan 3, 2020 •
edited

@jayconrod Sure, that makes sense. This is covered by #34506.

@kushthedude kushthedude mentioned this issue Sep 21, 2020
Merged
@Hipska Hipska mentioned this issue May 6, 2022
Closed
3 tasks
@Hipska
Copy link

Hipska commented May 17, 2022

Is there any update on this? https://github.com/influxdata/telegraf/blob/master/go.sum has 100+ duplicate lines..

@bcmills
Copy link
Contributor

bcmills commented May 19, 2022

@Hipska, this issue is still on the backlog. If you'd like to contribute a fix I'd be happy to review it!

@powersj powersj mentioned this issue Jun 8, 2022
Closed
@gopherbot
Copy link

Change https://go.dev/cl/411154 mentions this issue: cmd/go/internal/modfetch: prevent duplicate hashes in go.sum

@golang golang locked and limited conversation to collaborators Jun 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

Successfully merging a pull request may close this issue.

cmd/go/internal/modfetch: prevent duplicate hashes in go.sum powersj/go
9 participants
@rsc @honnix @jayconrod @Hipska @andybons @tv42 @myitcv @bcmills @gopherbot