Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: downgrading a module with go get retains indirect dependencies #26481

Closed
mwf opened this issue Jul 19, 2018 · 4 comments
Closed

cmd/go: downgrading a module with go get retains indirect dependencies #26481

mwf opened this issue Jul 19, 2018 · 4 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@mwf
Copy link

mwf commented Jul 19, 2018

What version of Go are you using (go version)?

Latest go devel: go version devel +d278f09333 Thu Jul 19 05:40:37 2018 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ikorolev/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ikorolev/.gvm/pkgsets/go1.11beta1/global:/Users/ikorolev/dev/go"
GOPROXY=""
GORACE=""
GOROOT="/Users/ikorolev/.gvm/gos/go1.11beta1"
GOTMPDIR=""
GOTOOLDIR="/Users/ikorolev/.gvm/gos/go1.11beta1/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/go-build301560759=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

It's maybe connected with #26474 but is much more severe.

Sorry, can't give you a standalone reproduction again - bug is produced only with downloading updated dependencies.
Consider the following flow:

C(v0.1.0) uses A(v0.1.0):

module github.com/mwf/vgo-modules/c

require github.com/mwf/vgo-modules/a v0.1.0

Then A(v0.2.0) is released and C(v0.2.0) depending on new A(v0.2.0).

We got a project main which uses C(v0.1.0) and we want to upgrade C to v0.2.0

cd `mktemp -d`
git clone https://github.com/mwf/vgo-modules .
cd c_user
echo "\n$(cat go.mod)"
echo "\n$(cat go.sum)"
go get github.com/mwf/vgo-modules/c@v0.2.0
echo "\n$(cat go.mod)"

Output:

module github.com/mwf/vgo-modules/c_user

require github.com/mwf/vgo-modules/c v0.1.0

github.com/mwf/vgo-modules/a v0.1.0 h1:QD+ijrXwAYk4CMTOqEAzcogJoF4zLfit2alZVmT80EM=
github.com/mwf/vgo-modules/a v0.1.0/go.mod h1:XGJvSKC62ygHgRNmDf4RqXyp0zngXIJMLc6BaSfyTfI=
github.com/mwf/vgo-modules/c v0.1.0 h1:db5PK5vzxlLviLcUsd1YCaLjG3+35hw6OMKSdVzqPMo=
github.com/mwf/vgo-modules/c v0.1.0/go.mod h1:qfiW4sL7v2Bnu6kBwvwxeVylzuPa6cxOldlsx2NCIRI=

module github.com/mwf/vgo-modules/c_user

require github.com/mwf/vgo-modules/c v0.2.0

All seems OK, than we run tests and understand that C(v0.2.0) is broken - go test .

--- FAIL: TestC (0.00s)
    main_test.go:12: c.CA: got CAaaaa, expected: CA
FAIL
FAIL    github.com/mwf/vgo-modules/c_user   0.014s

We decide to downgrade C back to v0.1.0 used before:

go get github.com/mwf/vgo-modules/c@v0.1.0
echo "\n$(cat go.mod)"
go test .

What did you expect to see?

go.mod should contain downgraded c dependency only

module github.com/mwf/vgo-modules/c_user

require github.com/mwf/vgo-modules/c v0.1.0

What did you see instead?

After the downgrade we get unexpected github.com/mwf/vgo-modules/a v0.2.0 // indirect and tests still fail.

module github.com/mwf/vgo-modules/c_user

require (
    github.com/mwf/vgo-modules/a v0.2.0 // indirect
    github.com/mwf/vgo-modules/c v0.1.0
)

--- FAIL: TestC (0.00s)
    main_test.go:12: c.CA: got CAaaaa, expected: CA
FAIL
FAIL    github.com/mwf/vgo-modules/c_user   0.018s

Some thoughts

We should definitely downgrade the transitive dependencies in this case.

But I understand it's a nontrivial task in general - because we should somehow distinguish cases when we intentionally add a v0.1.1 in main, then upgrade && downgrade c.

Ideally, we should go back to a v0.1.1 and c v.1.0, but to achieve this we should track the history somehow - and we don't have such feature using go.sum now.

cc @myitcv @rsc

@bcmills
Copy link
Contributor

bcmills commented Jul 19, 2018

Ideally, we should go back to a v0.1.1 and c v.1.0, but to achieve this we should track the history somehow

In the typical case you can just check out the previous go.mod from version control, right?

It gets to be more of a problem when you're upgrading and downgrading a bunch of unrelated dependencies in the same change, but that seems easy to avoid: how big is this problem in practice?
(Concrete examples of workflows that can lead to this kind of problem would be helpful.)

$ cd `mktemp -d`
$ git clone https://github.com/mwf/vgo-modules .
$ cd c_user

c_user$ go list -m all
go: finding github.com/mwf/vgo-modules/c v0.1.0
github.com/mwf/vgo-modules/c_user
github.com/mwf/vgo-modules/a v0.1.0
github.com/mwf/vgo-modules/c v0.1.0

c_user$ go get github.com/mwf/vgo-modules/c@v0.2.0
go: finding github.com/mwf/vgo-modules/c v0.2.0
go: finding github.com/mwf/vgo-modules/a v0.2.0
go: downloading github.com/mwf/vgo-modules/c v0.2.0
go: downloading github.com/mwf/vgo-modules/a v0.2.0

c_user$ go list -m all
github.com/mwf/vgo-modules/c_user
github.com/mwf/vgo-modules/a v0.2.0
github.com/mwf/vgo-modules/c v0.2.0

c_user$ git checkout HEAD go.mod

c_user$ go list -m all
github.com/mwf/vgo-modules/c_user
github.com/mwf/vgo-modules/a v0.1.0
github.com/mwf/vgo-modules/c v0.1.0

@bcmills bcmills changed the title cmd/go: downgrading a module with go get produces phantom indirect dependencies cmd/go: downgrading a module with go get retains indirect dependencies Jul 19, 2018
@mwf
Copy link
Author

mwf commented Jul 19, 2018

VCS checkout is great, but it helps only in trivial case when you update, run tests and determine the failure. Then you rollback with VCS and you're done.

But imagine more complex situation, when the bug is hidden - either it's not covered with tests properly or it's just too complex and happens in some particular circumstances in runtime. With a panic for example or even some slow goroutine or file descriptor leak.

So the gap between updating the go.mod and getting the bug could be hours-days. And even several application releases could happen in this time, with other dependency upgrades possible.

Imagine this bug is in production and all you want in this minute is a quick hotfix. You may try to use the VCS, checkout the dependency version from that commit and just take it.
But I bet it could be a hard task if this dependency has a big transitive dependency tree like https://github.com/coreos/etcd for example.

So in case of VCS checkout you will have to doublecheck the transitive dependencies in go.mod after go mod -sync.
I would prefer instead if go get <rollbacked version> does this job for me and takes precisely the transitive dependencies from the previous version, which were stably used.

And the current behaviour doesn't match with doc from go help module-get saying

Similarly, downgrading one dependency may
require downgrading other dependenceis, and 'go get' does
this automatically as well.

It doesn't downgrade the other dependencies now actually.

PS There is a typo in the docs :) dependenceis -> dependencies

@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Jul 20, 2018
@bcmills bcmills added this to the Go1.11 milestone Jul 20, 2018
@bcmills
Copy link
Contributor

bcmills commented Aug 3, 2018

I would prefer instead if go get <rollbacked version> does this job for me and takes precisely the transitive dependencies from the previous version, which were stably used.

Rolling back to a previous state is conceptually very different from go get. go get means “make the minimal change necessary to get to the requested version”, not “return me to a point in history in which I depended on that version”.

The go tool is not a version control system. It interprets the current contents of your go.mod file, not its complete history.

@rsc
Copy link
Contributor

rsc commented Aug 10, 2018

This is working as designed, as intended, and as documented: go get means make the exact, unique minimum number changes to effect that command, nothing more, nothing less. You are asking for go get to magically infer the "more" you want. That's just not possible.

You are asking for version control on go.mod. The go.mod file has been engineered to be version control-friendly, precisely so that job can be reasonably done with your version control system of choice, instead of having the go command reimplement a new version control system of its own.

Sorry, I know this is not the answer you want.

@rsc rsc closed this as completed Aug 10, 2018
@golang golang locked and limited conversation to collaborators Aug 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants