cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module #29773

myitcv · 2019-01-16T16:20:48Z

What version of Go are you using (`go version`)?

$ go version
go version go1.12beta2 linux/amd64

Does this issue reproduce with the latest release?

Testing with 1.12 pre the release.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPROXY=""
GORACE=""
GOROOT="/home/myitcv/gos"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/tmp.Z9hWoyAQkk/x/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build313729444=/tmp/go-build -gno-record-gcc-switches"

What did you do?

cd $(mktemp -d)
git clone https://github.com/myitcv/x
cd x
git checkout da37b5e
go mod tidy
git status --porcelain
go list honnef.co/go/js/util

This gives the output:

honnef.co/go/js/util

as expected.

However if we then:

cat <<EOD >> tools.go
import _ "github.com/myitcv/gobin"
EOD
go mod tidy

followed by:

go list honnef.co/go/js/util
go list honnef.co/go/js/util
go list honnef.co/go/js/util
...

any number of times, the output is always:

go: finding honnef.co/go/js/util latest
honnef.co/go/js/util

This despite the requirement being part of the requirements graph.

$ go mod graph | grep util
myitcv.io@v0.0.0-20181102104800-990e6f9524a1 honnef.co/go/js/util@v0.0.0-20150216223935-96b8dd9d1621

What did you expect to see?

Not to see see the "finding" message.

What did you see instead?

The "finding" message as part of each go list.

Could this be something to do with the now circular module requirement myitcv.io -> github.com/myitcv/gobin -> myitcv.io? And the fact:

$ go mod why honnef.co/go/js/util
myitcv.io/react/examples
honnef.co/go/js/xhr
honnef.co/go/js/util

where both the honnef.co requirements are not actual modules?

The addition of the the requirement on github.com/myitcv/gobin has removed the indirect honnef.co/go/js/util from myitcv.io's go.mod, but I think this is expected.

cc @bcmills. I note #27123 which links to #27102 and #27063; I don't think this is precisely an instance of any of those.

Apologies; I don't think this can be reproduced in a testscript test because this appears to rely on the remote import path resolution code path.

cc @rogpeppe as an FYI given his report in #27123

The text was updated successfully, but these errors were encountered:

bcmills · 2019-01-23T20:43:02Z

And go mod download -json exits with success without actually printing anything:

x$ go mod download -json honnef.co/go/js/util

x$

bcmills · 2019-01-23T22:26:54Z

I think that the problem here is a circular dependency via an older version of the main module. Probably has something to do with a hard-coded bypass for loading the target module. Minimal example forthcoming.

bcmills · 2019-01-24T22:43:47Z

Found a simpler repro.

bcmills · 2019-01-24T22:48:55Z

root$ mkdir -p ./mirror-v0.1.0 ./pkg-v0.1.0 ./root-v0.1.0

root$ cat >./go.mod <<EOF
module golang.org/issue/root

go 1.12

replace (
        golang.org/issue/mirror v0.1.0 => ./mirror-v0.1.0
        golang.org/issue/pkg v0.1.0 => ./pkg-v0.1.0
        golang.org/issue/root v0.1.0 => ./root-v0.1.0
)

require golang.org/issue/mirror v0.1.0
EOF

root$ cat >./root.go <<EOF
package root

import _ "golang.org/issue/mirror"
EOF

root$ cat >./mirror-v0.1.0/go.mod <<EOF
module golang.org/issue/mirror

require golang.org/issue/root v0.1.0
EOF

root$ cat >./mirror-v0.1.0/mirror.go <<EOF
package mirror

import _ "golang.org/issue/pkg"
EOF

root$ cat >./pkg-v0.1.0/go.mod <<EOF
module golang.org/issue/pkg
EOF

root$ cat >./pkg-v0.1.0/pkg.go <<EOF
package pkg
EOF

root$ cat >./root-v0.1.0/go.mod <<EOF
module golang.org/issue/root

require golang.org/issue/pkg v0.1.0
EOF

root$ go list -f '{{with .Module}}{{.Path}} {{.Version}}{{end}}' all
golang.org/issue/mirror v0.1.0
golang.org/issue/pkg v0.1.0
golang.org/issue/root

root$ go list -m all
golang.org/issue/root
golang.org/issue/mirror v0.1.0 => ./mirror-v0.1.0

root$ go mod graph
golang.org/issue/root golang.org/issue/mirror@v0.1.0
golang.org/issue/mirror@v0.1.0 golang.org/issue/root@v0.1.0
golang.org/issue/root@v0.1.0 golang.org/issue/pkg@v0.1.0

Because of golang/go#29773, circular module requirements create a problem with Go 1.11 and 1.12beta2 (at least). The fix for this is scheduled for Go 1.13, but we need to add a dependency from myitcv.io -> github.com/myitcv/gobin now. Given that the dependency on myitcv.io is only a development-time tool dependency, the easiest thing to do for now is to drop that requirement, thereby breaking the cycle.

myitcv · 2019-01-25T11:47:57Z

@bcmills - thanks very much for taking a look at this.

Just an FYI - it looks from the above like you have a standalone repro, correct? Also that this is going to be fixed in the 1.13 cycle?

If so, I'm going to break the cycle in this instance. The requirement github.com/myitcv/gobin -> myitcv.io is a development-time tool dependency... and not a critical one at that so I can live with this for now.

Just wanted to flag that I'll be doing that in case you're attempting any repros against master on either.

Because of golang/go#29773, circular module requirements create a problem with Go 1.11 and 1.12beta2 (at least). The fix for this is scheduled for Go 1.13, but we need to add a dependency from myitcv.io -> github.com/myitcv/gobin now. Given that the dependency on myitcv.io is only a development-time tool dependency, the easiest thing to do for now is to drop that requirement, thereby breaking the cycle.

bcmills · 2019-01-25T19:31:47Z

Yes, I have a standalone repro and I plan to have it fixed in 1.13. (I'm hoping that the fix will be straightforward, but as the Google SREs say, “hope is not a strategy”.)

Took me a while to distill it out, but the steps you gave did reproduce the problem exactly as described. Thanks for the detailed report!

gopherbot · 2019-07-17T17:05:18Z

Change https://golang.org/cl/186537 mentions this issue: cmd/go/internal/mvs: retain modules required by older versions

mvdan added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Jan 16, 2019

This comment has been minimized.

Sign in to view

bcmills changed the title ~~cmd/go: circular module requirement means non-module requirements are never satisfied~~ cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module Jan 24, 2019

bcmills added this to the Go1.13 milestone Jan 24, 2019

This comment has been minimized.

Sign in to view

myitcv mentioned this issue Jan 25, 2019

deps: temporarily remove requirement on myitcv.io myitcv/gobin#58

Merged

This was referenced Apr 4, 2019

cmd/go: mod tidy removes lines that build seems to need #31248

Closed

cmd/go: go mod change vendor content #31223

Closed

dmitshur added the GoCommand cmd/go label May 1, 2019

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

bcmills mentioned this issue Jul 17, 2019

cmd/go: heisen-dependency on google.golang.org/appengine #33151

Closed

bcmills self-assigned this Jul 17, 2019

bcmills modified the milestones: Go1.14, Go1.13 Jul 18, 2019

gopherbot closed this as completed in a005f99 Jul 18, 2019

bcmills mentioned this issue Sep 13, 2019

switch build scripts to use go modules, stop requiring $GOPATH kubernetes/kubernetes#82531

Open

10 tasks

bcmills mentioned this issue Oct 3, 2019

cmd/go: automatically check and use vendored packages #33848

Closed

ceejatec mentioned this issue Jun 13, 2020

cmd/go: go.mod inconsistency between "go mod tidy" and "go build" #39570

Closed

golang locked and limited conversation to collaborators Jul 17, 2020

gopherbot added the FrozenDueToAge label Jul 17, 2020

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module #29773

cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module #29773

myitcv commented Jan 16, 2019

This comment has been minimized.

bcmills commented Jan 23, 2019

This comment has been minimized.

bcmills commented Jan 23, 2019

This comment has been minimized.

This comment has been minimized.

bcmills commented Jan 24, 2019 •

edited

bcmills commented Jan 24, 2019 •

edited

myitcv commented Jan 25, 2019

bcmills commented Jan 25, 2019

gopherbot commented Jul 17, 2019

cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module #29773

cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module #29773

Comments

myitcv commented Jan 16, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

This comment has been minimized.

bcmills commented Jan 23, 2019

This comment has been minimized.

bcmills commented Jan 23, 2019

This comment has been minimized.

This comment has been minimized.

bcmills commented Jan 24, 2019 • edited

bcmills commented Jan 24, 2019 • edited

myitcv commented Jan 25, 2019

bcmills commented Jan 25, 2019

gopherbot commented Jul 17, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

bcmills commented Jan 24, 2019 •

edited

bcmills commented Jan 24, 2019 •

edited