cmd/go: mod tidy removes lines that build seems to need #31248

zombiezen · 2019-04-03T23:02:15Z

(I haven't dug into exactly what dependency is causing this or why, but I've managed to whittle the issue down to a fairly manageable no-code repro case.)

What version of Go are you using (`go version`)?

$ go version
go 1.12.1

Does this issue reproduce with the latest release?

Yes. I haven't checked against tip.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rlight2/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rlight2"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rlight2/src/mg-repro/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build158725968=/tmp/go-build -gno-record-gcc-switches"

What did you do?

git clone https://gist.github.com/5513fc51a179cfd757dcdd1d7c1e4995.git mod-tidy-issue
cd mod-tidy-issue
go mod tidy
go build -mod=readonly

main.go + go.mod gist

What did you expect to see?

A successful build of the binary.

What did you see instead?

go: updates to go.mod needed, disabled by -mod=readonly

Additional surrounding steps

Running go build will cause go build -mod=readonly to pass. Doing this adds lines (all marked "indirect") for:

github.com/golang/mock
gopkg.in/check.v1
gopkg.in/yaml.v2

However, running go mod tidy on this go.mod will unceremoniously remove these added lines. Most confusingly, go mod why doesn't seem to know why these lines are there either:

$ go mod why -m gopkg.in/yaml.v2
# gopkg.in/yaml.v2
(main module does not need module gopkg.in/yaml.v2)

The text was updated successfully, but these errors were encountered:

andybons · 2019-04-04T17:13:29Z

@bcmills @jayconrod

bcmills · 2019-04-04T22:56:55Z

We should check for similarities with #29773. (I don't know whether it will turn out to be related, but it's a known source of missing dependencies.)

prymitive · 2019-06-07T20:01:32Z

Hit the same issue - go build and go mod tidy keep fighting over what should be in go.mod
gist for reference - https://gist.github.com/prymitive/9f8ac72d9a88f1704efa8ebfdf133bd1

$ git st
On branch master
[...]
	new file:   go.mod
	new file:   go.sum
	new file:   main.go
$ go build
$ git diff
diff --git a/go.mod b/go.mod
index 6980ec7..7e79628 100644
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
        github.com/gin-gonic/contrib v0.0.0-20190526021735-7fb7810ed2a0
        github.com/gin-gonic/gin v1.4.0
        github.com/go-bindata/go-bindata v3.1.1+incompatible
+       github.com/go-logfmt/logfmt v0.4.0 // indirect
        github.com/go-openapi/errors v0.19.0
        github.com/go-openapi/runtime v0.19.0
        github.com/go-openapi/strfmt v0.19.0
$ go mod tidy
$ git diff
[no diff]
$ go build
$ git diff
diff --git a/go.mod b/go.mod
index 6980ec7..7e79628 100644
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
        github.com/gin-gonic/contrib v0.0.0-20190526021735-7fb7810ed2a0
        github.com/gin-gonic/gin v1.4.0
        github.com/go-bindata/go-bindata v3.1.1+incompatible
+       github.com/go-logfmt/logfmt v0.4.0 // indirect
        github.com/go-openapi/errors v0.19.0
        github.com/go-openapi/runtime v0.19.0
        github.com/go-openapi/strfmt v0.19.0

go.mod keeps getting updated on build but go mod tidy reverts these changes See golang/go#31248 (comment)

bcmills · 2019-06-07T21:01:49Z

@prymitive, can you provide a more compact example, or at least compare the output of go list -m all and go mod graph? As noted above, it would be good to rule out duplicates of #29773.

prymitive · 2019-06-08T10:39:47Z

Narrowed down needed modules to just two:

github.com/prometheus/client_golang v0.9.4
github.com/spf13/viper v1.4.0

https://gist.github.com/prymitive/9f8ac72d9a88f1704efa8ebfdf133bd1 updated to reflect that
Also added the output of these 2 commands

gopherbot · 2019-07-17T17:05:19Z

Change https://golang.org/cl/186537 mentions this issue: cmd/go/internal/mvs: retain modules required by older versions

gopherbot · 2019-07-17T18:38:49Z

Change https://golang.org/cl/186557 mentions this issue: cmd/go/internal/mvs: in Req, omit versions implied by older-than-selected versions already in the graph

bcmills · 2019-07-17T19:09:51Z

I found a minimal repro. A consistent reproducer requires two modules that depend on some module X. One of the modules must depend on the newer version of X and on the other module. The other module must depend on the older X, which must pull in some indirect dependency I that the newer X does not. Then mvs.Req normally adds I to the build list, but since I is not required by the selected versions and is also not needed for the import graph, go mod tidy removes it.

https://go-review.googlesource.com/c/go/+/186557/2/src/cmd/go/testdata/script/mod_indirect_tidy.txt

Fixes #29773 Updates #31248 Change-Id: Ic1923119c8cf3a60c586df1b270c3af0c9095f29 Reviewed-on: https://go-review.googlesource.com/c/go/+/186537 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

jgiles · 2019-08-06T22:36:57Z

I'm seeing an issue that looks like this on go1.12.7 . Is there anything I can do to confirm that I have a repro of this issue, or if I should file a separate bug?

horgh · 2019-08-06T23:45:06Z

You could see if the behaviour you're seeing is fixed using gotip as described here: #33435 (comment)

vibridi · 2019-10-23T15:48:23Z

@bcmills
Hello, sorry to revive a closed issue. I think I'm observing the same problem described here in Go1.13.3
In particular, go get disagrees with go mod tidy over which package should go into go.mod. In my case:

go get writes github.com/ugorji/go v1.1.7 // indirect, also

$ go mod why github.com/ugorji/go
# github.com/ugorji/go
(main module does not need package github.com/ugorji/go)

go mod tidy writes github.com/ugorji/go/codec v1.1.7 // indirect, also

$ go mod why github.com/ugorji/go/codec
# github.com/ugorji/go/codec
mycompany/mypackage
github.com/gin-gonic/gin
github.com/gin-gonic/gin/binding
github.com/ugorji/go/codec

bcmills · 2019-10-23T16:08:19Z

In particular, go get disagrees with go mod tidy over which package should go into go.mod.

That is #34086, fixed at head.

vibridi · 2019-10-24T11:16:34Z

Thank you very much

thepudds · 2019-10-24T11:51:42Z

@vibridi If you haven’t already done so, would you mind confirming the behavior you reported is indeed fixed by using the gotip command:

https://godoc.org/golang.org/dl/gotip

vibridi · 2019-10-24T23:58:37Z

@thepudds confirmed, it is fixed at head (680ed10) 👍

ceejatec · 2020-06-11T07:24:31Z

@bcmills Sorry to necropost, but I'm seeing a similar issue even with go 1.14.4. I have a package where "go mod tidy" removes an //indirect entry from go.mod, but that then causes "go build -mod=readonly" to fail with "cannot find module providing package github.com/dustin/gojson: import lookup disabled by -mod=readonly". On the other hand, running "go build" without "-mod=readonly" adds the same //indirect entries back into go.mod that "go mod tidy" just took out.

Before I attempt to reduce this to a test case and file a new ticket, could you tell me if the following statement should 100% be true?

If you run "go mod tidy", then immediately running "go build -mod=readonly"
should always succeed. (Well, at least if the code compiles -
it should not fail due to needing to write to go.mod or for other
reasons implied by -mod=readonly.)

If it isn't supposed to be 100% true, is there any statement that can be made about "go mod tidy" vs. "go build -mod=readonly" that is 100% true?

ceejatec · 2020-06-11T08:06:25Z

FYI, I also confirmed this behaviour running "gotip".

jayconrod · 2020-06-11T14:29:49Z

@ceejatec Your statement should be true. Please open a new issue, and we'll take a look.

ceejatec · 2020-06-13T00:39:57Z

@jayconrod Thanks for confirming. I've filed a new ticket #39570 with a reproducer.

zombiezen added the modules label Apr 3, 2019

andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 4, 2019

andybons added this to the Unplanned milestone Apr 4, 2019

jayconrod self-assigned this Apr 4, 2019

dmitshur added the GoCommand cmd/go label May 1, 2019

dmitshur mentioned this issue May 15, 2019

cmd/go: go list has too many (more than zero) side effects #29452

Closed

prymitive mentioned this issue Jun 7, 2019

chore(deps): update module prometheus/client_golang to v0.9.4 prymitive/karma#762

Merged

1 task

prymitive added a commit to prymitive/karma that referenced this issue Jun 7, 2019

fix(backend): add a workaround for Go modules issue

8117000

go.mod keeps getting updated on build but go mod tidy reverts these changes See golang/go#31248 (comment)

bcmills mentioned this issue Jul 17, 2019

cmd/go: heisen-dependency on google.golang.org/appengine #33151

Closed

bcmills assigned bcmills and unassigned jayconrod Jul 17, 2019

bcmills modified the milestones: Unplanned, Go1.13 Jul 18, 2019

gopherbot closed this as completed in 52fdd62 Jul 18, 2019

bcmills mentioned this issue Aug 2, 2019

cmd/go: go mod vendor changes go.mod #33435

Closed

This was referenced Sep 4, 2019

cmd/go: module project cannot update go.sum file in readonly mode #34054

Closed

Go 1.12.5 and 1.13 conflict over go.mod #34135

Closed

zikaeroh mentioned this issue Sep 13, 2019

cmd/go: go adds unnecessary module dependency #34290

Closed

bcmills mentioned this issue Oct 3, 2019

cmd/go: automatically check and use vendored packages #33848

Closed

bcmills mentioned this issue Oct 15, 2019

cmd/go: decide replace, exclude wildcard behavior #26344

Closed

cofyc mentioned this issue Nov 1, 2019

go build/test adds new line which should not exist in go.mod pingcap/tidb-operator#1095

Closed

ceejatec mentioned this issue Jun 13, 2020

cmd/go: go.mod inconsistency between "go mod tidy" and "go build" #39570

Closed

golang locked and limited conversation to collaborators Jun 13, 2021

gopherbot added the FrozenDueToAge label Jun 13, 2021

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: mod tidy removes lines that build seems to need #31248

cmd/go: mod tidy removes lines that build seems to need #31248

zombiezen commented Apr 3, 2019

andybons commented Apr 4, 2019

bcmills commented Apr 4, 2019

prymitive commented Jun 7, 2019

bcmills commented Jun 7, 2019

prymitive commented Jun 8, 2019

gopherbot commented Jul 17, 2019

gopherbot commented Jul 17, 2019

bcmills commented Jul 17, 2019 •

edited

jgiles commented Aug 6, 2019

horgh commented Aug 6, 2019

vibridi commented Oct 23, 2019

bcmills commented Oct 23, 2019

vibridi commented Oct 24, 2019

thepudds commented Oct 24, 2019

vibridi commented Oct 24, 2019

ceejatec commented Jun 11, 2020 •

edited

ceejatec commented Jun 11, 2020

jayconrod commented Jun 11, 2020

ceejatec commented Jun 13, 2020 •

edited by dmitshur

cmd/go: mod tidy removes lines that build seems to need #31248

cmd/go: mod tidy removes lines that build seems to need #31248

Comments

zombiezen commented Apr 3, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

Additional surrounding steps

andybons commented Apr 4, 2019

bcmills commented Apr 4, 2019

prymitive commented Jun 7, 2019

bcmills commented Jun 7, 2019

prymitive commented Jun 8, 2019

gopherbot commented Jul 17, 2019

gopherbot commented Jul 17, 2019

bcmills commented Jul 17, 2019 • edited

jgiles commented Aug 6, 2019

horgh commented Aug 6, 2019

vibridi commented Oct 23, 2019

bcmills commented Oct 23, 2019

vibridi commented Oct 24, 2019

thepudds commented Oct 24, 2019

vibridi commented Oct 24, 2019

ceejatec commented Jun 11, 2020 • edited

ceejatec commented Jun 11, 2020

jayconrod commented Jun 11, 2020

ceejatec commented Jun 13, 2020 • edited by dmitshur

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

bcmills commented Jul 17, 2019 •

edited

ceejatec commented Jun 11, 2020 •

edited

ceejatec commented Jun 13, 2020 •

edited by dmitshur