cmd/go: make `go mod download` with no arguments leave go.sum alone #45332

aofei · 2021-04-01T13:31:10Z

What version of Go are you using (`go version`)?

$ go version
go version go1.16.2 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/home/aofei/bin"
GOCACHE="/home/aofei/.cache/go-build"
GOENV="/home/aofei/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/aofei/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/aofei/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1210455100=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ git clone -b v1.1.5 https://github.com/aofei/sandid.git
$ cd sandid
$ go mod download
$ git diff
diff --git a/go.sum b/go.sum
index c221f64..ec9d676 100644
--- a/go.sum
+++ b/go.sum
@@ -3,6 +3,7 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0 h1:4G4v2dO3VZwixGIRoQ5Lfboy6nUhCyYzaqnIAPPhYs4=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
$ go mod tidy
$ git status
Not currently on any branch.
nothing to commit, working tree clean

What did you expect to see?

I expect to see that go mod download doesn't change anything. Or, at least it shouldn't fight with go mod tidy.

What did you see instead?

It adds unnecessary entries to go.sum.

A possible solution is to modload.TrimGoSum before modload.WriteGoMod(ctx).

Or, maybe go mod download shouldn't update go.mod or go.sum at all. Maybe people just expect it to fill the local cache. So another possible solution is to remove modload.WriteGoMod(ctx).

@jayconrod @bcmills @matloob

The text was updated successfully, but these errors were encountered:

aofei · 2021-04-01T13:33:01Z

@gopherbot, please add labels GoCommand, modules

jayconrod · 2021-04-01T14:12:11Z

This was an intentional change in 1.16, but I don't think we should close this issue unless we're sure this is the behavior we want.

Previously, go mod download update go.mod and go.sum after loading the build list, but it didn't update go.sum after downloading modules. It seems like it should do both or neither (like -mod=readonly or -mod=mod essentially, though go mod download does not accept the -mod flag).

One particular problem we were trying to solve: with the switch to -mod=readonly was that in some cases, you need the sum for a module's content, even if you don't need any packages from it. That happens when a module in the build list has a path that's a prefix of another module. When the go command loads a package from one, it needs to confirm the package is not also in the other. For example, the package example.com/a/b could be in either module example.com/a or example.com/a/b, and we need to load both to be sure.

If one of the sums is missing, we need a command more focused than go mod tidy to add the sum. go mod download example.com/a does that now.

We've gotten a fair amount of feedback that this isn't what people expect when running go mod download without arguments. go mod download doesn't load the package graph, so it doesn't know whether go mod tidy would keep a sum or not. Maybe it just shouldn't write go.sum unless a specific module is requested.

jayconrod · 2021-04-01T14:15:10Z

Incidentally, until this issue is resolved, I'd recommend go list -test -deps ./... instead of go mod download for populating the module cache.

ianlancetaylor · 2021-04-19T22:44:58Z

Marking as release blocker to force a decision for 1.17.

toothrot · 2021-04-29T18:13:51Z

@jayconrod Do you have any more thoughts on this? Checking in as it is labeled a release-blocker for Go 1.17.

rsc · 2021-05-04T17:15:48Z

@jayconrod suggests that 'go mod download' with no arguments be defined to never edit go.sum.
That sounds reasonable for Go 1.17.

gopherbot · 2021-05-10T22:15:42Z

Change https://golang.org/cl/318629 mentions this issue: cmd/go: don't update go.sum in 'go mod download' without args

gopherbot · 2021-05-11T15:45:24Z

Change https://golang.org/cl/318811 mentions this issue: all: update tests to use 'go mod download all' to populate go.sum

In anticipation of CL 318629, 'go mod download' without arguments will not update go.mod or go.sum. Before 1.16, 'go mod download' would adds sums for .mod files but not .zip files (which people didn't usually notice). Many folks found the behavior of adding sums for .zip files to be annoying. This change alters tests to run 'go mod download all' to populate go.sum files. This is equivalent to 'go mod download' without arguments before CL 318629. For golang/go#45332 Change-Id: I387d514176f798ae8f17b0b056194196718f57f5 Reviewed-on: https://go-review.googlesource.com/c/tools/+/318811 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> gopls-CI: kokoro <noreply+kokoro@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>

jayconrod · 2021-05-17T19:43:08Z

@gopherbot Please backport to 1.16. We plan to revert to the 1.15 behavior.

gopherbot · 2021-05-17T19:43:23Z

Backport issue(s) opened: ~~#46213 (for 1.15),~~ #46214 (for 1.16).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

dmitshur · 2021-05-20T18:26:38Z

@jayconrod This is still open, checking what is there still left to do here? Thanks.

jayconrod · 2021-05-21T13:05:24Z

The fix is ready, but I'm waiting on a review.

gopherbot · 2021-05-21T20:53:34Z

Change https://golang.org/cl/321956 mentions this issue: cmd/go: in TestScript/mod_replace, download an explicit module path

As of CL 318629, 'go mod download' without arguments does not save checksums for module source code. Without a checksum, 'go list' will not report the location of the source code even if it is present, in order to prevent accidental access of mismatched code. Downloading an explicit module here also more clearly expresses the intent of the test (“download this module and see where it is”), and may be somewhat more efficient (since the test doesn't need source code for the other modules in the build list). Updates #45332 Change-Id: Ic589b22478e3ed140b95365bb6729101dd598ccc Reviewed-on: https://go-review.googlesource.com/c/go/+/321956 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>

gopherbot · 2021-05-24T20:59:59Z

Change https://golang.org/cl/322369 mentions this issue: cmd/go: don't let 'go mod download' save sums for inconsistent requirements

…ements 'go mod download' calls modload.LoadModFile early to find the main module path in order to validate arguments. LoadModFile may write go.mod and go.sum to fix formatting and add a go directive. This calls keepSums, which, in eager mode, loaded the complete module graph in order to find out what sums are needed to load the complete module graph. If go.mod requires a lower version of a module than will be selected later, keepSums causes the sum for that version's go.mod to be retained, even though it isn't needed later after a consistent go.mod is written. This CL fixes keepSums not to load the graph if it hasn't already been loaded (whether eager or lazy), addressing comments from CL 318629. For #45332 Change-Id: I20d4404004e4ad335450fd0fd753e7bc0060f702 Reviewed-on: https://go-review.googlesource.com/c/go/+/322369 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org>

In anticipation of CL 318629, 'go mod download' without arguments will not update go.mod or go.sum. Before 1.16, 'go mod download' would adds sums for .mod files but not .zip files (which people didn't usually notice). Many folks found the behavior of adding sums for .zip files to be annoying. This change alters tests to run 'go mod download all' to populate go.sum files. This is equivalent to 'go mod download' without arguments before CL 318629. For golang/go#45332 Change-Id: I387d514176f798ae8f17b0b056194196718f57f5 Reviewed-on: https://go-review.googlesource.com/c/tools/+/318811 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> gopls-CI: kokoro <noreply+kokoro@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>

gopherbot added GoCommand cmd/go modules labels Apr 1, 2021

jayconrod added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Apr 1, 2021

jayconrod modified the milestones: go1.17, Go1.17 Apr 1, 2021

ianlancetaylor added the release-blocker label Apr 19, 2021

rsc changed the title ~~cmd/go: go mod download adds unnecessary entries to go.sum~~ cmd/go: make go mod download with no arguments leave go.sum alone May 4, 2021

rsc added the NeedsFix The path to resolution is known, but the work has not been done. label May 4, 2021

gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label May 4, 2021

jayconrod self-assigned this May 5, 2021

This was referenced May 17, 2021

cmd/go: make go mod download with no arguments leave go.sum alone [1.15 backport] #46213

Closed

cmd/go: make go mod download with no arguments leave go.sum alone [1.16 backport] #46214

Closed

bcmills mentioned this issue May 18, 2021

x/build/cmd/gopherbot: ignore versions in other paragraphs of backport requests #27489

Open

gopherbot closed this as completed in 4fb10b2 May 21, 2021

bcmills mentioned this issue Jun 2, 2021

cmd/go: unhelpful error message when running "go install" on a replaced-but-not-required package #46528

Closed

bayandin mentioned this issue Jun 7, 2021

go 1.16.5 Homebrew/homebrew-core#78666

Closed

luhring mentioned this issue Jul 1, 2021

Pipelines allow unclean go.sum files that block our release pipeline anchore/grype#358

Closed

2 tasks

jeffwidman mentioned this issue Jan 3, 2022

cmd/go: mod download and go mod tidy conflict (1.16beta1 -> 1.16rc1 regression) #43994

Closed

golang locked and limited conversation to collaborators May 24, 2022

gopherbot added the FrozenDueToAge label May 24, 2022

rsc unassigned jayconrod Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: make `go mod download` with no arguments leave go.sum alone #45332

cmd/go: make `go mod download` with no arguments leave go.sum alone #45332

aofei commented Apr 1, 2021

aofei commented Apr 1, 2021

jayconrod commented Apr 1, 2021

jayconrod commented Apr 1, 2021

ianlancetaylor commented Apr 19, 2021

toothrot commented Apr 29, 2021

rsc commented May 4, 2021

gopherbot commented May 10, 2021

gopherbot commented May 11, 2021

jayconrod commented May 17, 2021

gopherbot commented May 17, 2021 •

edited by dmitshur

dmitshur commented May 20, 2021

jayconrod commented May 21, 2021

gopherbot commented May 21, 2021

gopherbot commented May 24, 2021

cmd/go: make go mod download with no arguments leave go.sum alone #45332

cmd/go: make go mod download with no arguments leave go.sum alone #45332

Comments

aofei commented Apr 1, 2021

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

aofei commented Apr 1, 2021

jayconrod commented Apr 1, 2021

jayconrod commented Apr 1, 2021

ianlancetaylor commented Apr 19, 2021

toothrot commented Apr 29, 2021

rsc commented May 4, 2021

gopherbot commented May 10, 2021

gopherbot commented May 11, 2021

jayconrod commented May 17, 2021

gopherbot commented May 17, 2021 • edited by dmitshur

dmitshur commented May 20, 2021

jayconrod commented May 21, 2021

gopherbot commented May 21, 2021

gopherbot commented May 24, 2021

cmd/go: make `go mod download` with no arguments leave go.sum alone #45332

cmd/go: make `go mod download` with no arguments leave go.sum alone #45332

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

gopherbot commented May 17, 2021 •

edited by dmitshur