Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go test ./... or go build should not update existing dependencies and only add new dependencies as needed to satisfy imports #37780

Closed
gogrisohil opened this issue Mar 10, 2020 · 4 comments
Closed

cmd/go: go test ./... or go build should not update existing dependencies and only add new dependencies as needed to satisfy imports #37780

gogrisohil opened this issue Mar 10, 2020 · 4 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@gogrisohil
Copy link

gogrisohil commented Mar 10, 2020 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.14 darwin/amd64

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/username/Library/Caches/go-build"
GOENV="/Users/username/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="github.com/goguardian/Development"
GONOSUMDB="github.com/goguardian/Development"
GOOS="darwin"
GOPATH="/some-directory/golang"
GOPRIVATE="our-repo"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/some-directory/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/fx/h3bdzgd96mbc04y4ncqsfpw00000gn/T/go-build022976900=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Running go test ./... or go build on our CI systems tries to update the go.mod and go.sum. When I tried running go mod tidy it updated 2 existing dependencies in the go.mod:

updates github.com/grpc-ecosystem/grpc-gateway from v1.14.1 to v1.14.2 
updates google.golang.org/genproto from v0.0.0-20200306153348-d950eab6f860 to v0.0.0-20200309141739-5b75447e413d

It also adds lines in go.sum for the new packages

github.com/grpc-ecosystem/grpc-gateway v1.14.2 h1:SG3eXGmMVahaP4UtKsO/gPQpkovjXOmxXNd7sJlhxNs=
github.com/grpc-ecosystem/grpc-gateway v1.14.2/go.mod h1:6CwZWGDSPRJidgKAtJVvND6soZe6fT7iteq8wDPdhb0=

google.golang.org/genproto v0.0.0-20200309141739-5b75447e413d h1:VQ0pz4dAUaMWcQLM7tGY8Nk691kSrlGPyF5nSgAIw2g=
google.golang.org/genproto v0.0.0-20200309141739-5b75447e413d/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=

What did you expect to see?

Make sure all imports are satisfied by existing dependencies present in the go.mod, go.sum

What did you see instead?

Tries to update existing dependencies
@gogrisohil gogrisohil changed the title cmd/go: go test ./... should not update existing dependencies and only add new ones cmd/go: go test ./... should not update existing dependencies and only add new dependencies as needed to satisfy imports Mar 10, 2020
@gogrisohil gogrisohil changed the title cmd/go: go test ./... should not update existing dependencies and only add new dependencies as needed to satisfy imports cmd/go: go test ./... or go build should not update existing dependencies and only add new dependencies as needed to satisfy imports Mar 10, 2020
@jayconrod
Copy link
Contributor

Build commands like go build ./... and go test ./... will add a new module requirement to go.mod if a package is imported that is not provided by any currently required module. The new requirement will be added at the latest version. If the newly required module requires higher versions of modules that you already depend on, those modules will be updated, too. If your module is "tidy" already (go mod tidy makes no changes), then go test ./... should not change go.mod.

If you don't see any new requirement being added, it's possible that your go.mod file is not consistent with the versions actually being used. For example, if you depend on A at v1.0.0, and you depend on B at v1.0.0, but B depends on A at v1.5.0, then you'll actually end up using A at v1.5.0. Commands that load the module graph will update go.mod to ensure that requirements are accurate.

Could you confirm one of these situations is what's happening? If not, could you post a small reproducible example?

Also, if you don't want dependencies to be updated in either situation, have you tried using the -mod=readonly flag? It does exactly this.

@jayconrod jayconrod added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Mar 11, 2020
@gogrisohil
Copy link
Author

gogrisohil commented Mar 11, 2020 •
edited

Could you confirm one of these situations is what's happening?

I have not looked at the dependency and transitive dependency versions into much detail because we have a lot of them. I am not sure if there's a way to run go commands in debug or verbose mode to try to figure out why it's insisting on updating dependencies so we can look into that.

have you tried using the -mod=readonly flag

We tried that but the builds on our CI failed due to this error go: updates to go.mod needed, disabled by -mod=readonly. That being said, if we run go mod tidy, it does update the dependencies in the go.mod and go.sum as I mentioned in the description earlier and then the builds pass.

But my question is that is there a way to make go services build with existing set of dependencies in the go.mod and go.sum files without running go mod tidy every so often when it is trying to update existing dependencies rather than adding new ones ?

@jayconrod
Copy link
Contributor

If a build is adding or updating dependencies, that means go.mod is incomplete or inconsistent. go mod tidy will make it complete and consistent, so I'd definitely recommend that. You can build with a partial set of dependencies, but the build command will update go.mod to ensure that builds will use the same set of versions in the future.

@gogrisohil
Copy link
Author

gogrisohil commented Mar 11, 2020 •
edited

Gotcha. I guess we will have to run go mod tidy every so often to stay updated with the latest dependencies. Closing the ticket.

@golang golang locked and limited conversation to collaborators Mar 11, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jayconrod @gopherbot @gogrisohil