Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: get panics with "can't find reason for requirement on" #47979

Closed
ash2k opened this issue Aug 26, 2021 · 20 comments
Closed

cmd/go: get panics with "can't find reason for requirement on" #47979

ash2k opened this issue Aug 26, 2021 · 20 comments
Labels
GoCommand cmd/go modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@ash2k
Copy link
Contributor

ash2k commented Aug 26, 2021

What version of Go are you using (go version)?

$ go version
go version go1.17 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/mikhail/Library/Caches/go-build"
GOENV="/Users/mikhail/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/mikhail/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/mikhail/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/opt/go/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/mikhail/src/gitlab-agent/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f5/v0r6vhks50d61jjmrhfgnlbh0000gn/T/go-build1390498625=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I'm trying new pruned modules functionality to see if it helps reduce the number of tracked dependencies in my project.

  • I've changed one of the libraries to go 1.17 and ran go1.17 mod tidy -compat=1.17. I had to change the go directive in go.mod to 1.17 manually, re-run the command, and only then it worked - is this expected? Pushed this into a branch (commit).
  • Then in my project I did the same as above, and then ran:
go1.17 get gitlab.com/gitlab-org/labkit@fc0c07abad6a32b5b77e16289fcc72d38f8411ab
go: downloading gitlab.com/gitlab-org/labkit v1.9.1-0.20210826103322-fc0c07abad6a
panic: internal error: can't find reason for requirement on github.com/google/gofuzz@v1.1.0

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc000ac4a68, 0xc00065d920}, {0xc00002afd0, 0xc000010570}})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1794 +0x10a
cmd/go/internal/modget.(*resolver).updateBuildList(0xc0005cc100, {0x166fac0, 0xc00002a0c0}, {0x0, 0x0, 0x0})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1799 +0x5bb
cmd/go/internal/modget.(*resolver).resolveQueries(0xc0005cc100, {0x166fac0, 0xc00002a0c0}, {0xc000010050, 0x1, 0x1ac4658})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1278 +0x1e5
cmd/go/internal/modget.runGet({0x166fac0, 0xc00002a0c0}, 0xc000026318, {0xc0000201d0, 0x1, 0x1})
        /usr/local/go/src/cmd/go/internal/modget/get.go:300 +0x374
main.invoke(0x1983c80, {0xc0000201c0, 0x2, 0x2})
        /usr/local/go/src/cmd/go/main.go:216 +0x2f6
main.main()
        /usr/local/go/src/cmd/go/main.go:173 +0x78e
  • Then I thought "It shouldn't crash, but maybe I've messed something up in the library repo?". I went there, ran go1.17 mod tidy (without -compat=1.17) and indeed there was a new change (commit). I pushed it too.
  • Ran go get in my project:
go1.17 get gitlab.com/gitlab-org/labkit@c3248dffceabec66f9f26a08feb847a438d33707
go: downloading gitlab.com/gitlab-org/labkit v1.9.1-0.20210826104146-c3248dffceab
panic: internal error: can't find reason for requirement on github.com/google/gofuzz@v1.1.0

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc0004c74d0, 0xc00073ed20}, {0xc00002afd6, 0xc000604090}})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1794 +0x10a
cmd/go/internal/modget.(*resolver).updateBuildList(0xc00059a100, {0x166fac0, 0xc00002a0c0}, {0x0, 0x0, 0x0})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1799 +0x5bb
cmd/go/internal/modget.(*resolver).resolveQueries(0xc00059a100, {0x166fac0, 0xc00002a0c0}, {0xc000010050, 0x1, 0x1ac4658})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1278 +0x1e5
cmd/go/internal/modget.runGet({0x166fac0, 0xc00002a0c0}, 0xc000026318, {0xc0000201d0, 0x1, 0x1})
        /usr/local/go/src/cmd/go/internal/modget/get.go:300 +0x374
main.invoke(0x1983c80, {0xc0000201c0, 0x2, 0x2})
        /usr/local/go/src/cmd/go/main.go:216 +0x2f6
main.main()
        /usr/local/go/src/cmd/go/main.go:173 +0x78e

The second time it actually was not crashing immediately but after a few seconds (looks like it downloaded the library this time) it still did.

Here is the commit in my project, on which I'm trying to run the above commands and it crashes.

What did you expect to see?

No crashes. If something is wrong with my modules or their dependencies, a comprehensible error.

What did you see instead?

See above.

@seankhliao seankhliao changed the title Go get panics with module pruning cmd/go: get panics with "can't find reason for requirement on" Aug 26, 2021
@seankhliao seankhliao added GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 26, 2021
@seankhliao
Copy link
Member

cc @bcmills @matloob @jayconrod

@bcmills bcmills self-assigned this Aug 26, 2021
@bcmills bcmills added this to the Go1.18 milestone Aug 26, 2021
@thomaspeugeot
Copy link

thomaspeugeot commented Aug 27, 2021

FYI, I met an identical issue.

thomaspeugeot@MacBook-Pro-de-Thomas gongsvg % go version
go version go1.17 darwin/amd64

to reproduce the issue:

git clone https://github.com/fullstack-lang/gongsvg
cd gongsvg
go get -d github.com/fullstack-lang/gongdoc@HEAD

panic

> Executing task: go get -d github.com/fullstack-lang/gongdoc@HEAD <

panic: internal error: can't find reason for requirement on gonum.org/v1/plot@v0.8.1

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc000814408, 0xc0003b6858}, {0xc0000b2da6, 0xc00028c480}})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1794 +0x10a
cmd/go/internal/modget.(*resolver).updateBuildList(0xc0002de300, {0x166fac0, 0xc0000b2000}, {0x0, 0x0, 0x0})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1799 +0x5bb
cmd/go/internal/modget.(*resolver).resolveQueries(0xc0002de300, {0x166fac0, 0xc0000b2000}, {0xc0000c0048, 0x1, 0x1ab7040})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1278 +0x1e5
cmd/go/internal/modget.runGet({0x166fac0, 0xc0000b2000}, 0xc0000da420, {0xc000098070, 0x1, 0x1})
        /usr/local/go/src/cmd/go/internal/modget/get.go:300 +0x374
main.invoke(0x1983c80, {0xc000098050, 0x3, 0x3})
        /usr/local/go/src/cmd/go/main.go:216 +0x2f6
main.main()
        /usr/local/go/src/cmd/go/main.go:173 +0x78e
The terminal process "zsh '-c', 'go get -d github.com/fullstack-lang/gongdoc@HEAD'" terminated with exit code: 2.

@bcmills
Copy link
Contributor

bcmills commented Aug 30, 2021

@ash2k, thanks for the detailed steps to reproduce! I can reproduce the panic with a go command built from head in your repo at the named commit:

~/tmp/gitlab-agent$ go get gitlab.com/gitlab-org/labkit@fc0c07abad6a32b5b77e16289fcc72d38f8411ab
go: downloading gitlab.com/gitlab-org/labkit v1.9.1-0.20210826103322-fc0c07abad6a
panic: internal error: can't find reason for requirement on github.com/google/gofuzz@v1.1.0

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc00067f6e0, 0xc0002ee948}, {0xc0000b8d86, 0xc000684a18}})
        /usr/local/google/home/bcmills/go/src/cmd/go/internal/modget/get.go:1810 +0x114
cmd/go/internal/modget.(*resolver).updateBuildList(0xc00052e100, {0xa82780, 0xc0000b8000}, {0x0, 0x0, 0x0})
        /usr/local/google/home/bcmills/go/src/cmd/go/internal/modget/get.go:1815 +0x59b
cmd/go/internal/modget.(*resolver).resolveQueries(0xc00052e100, {0xa82780, 0xc0000b8000}, {0xc0000c6048, 0x1, 0x1010000004e4ea6})
        /usr/local/google/home/bcmills/go/src/cmd/go/internal/modget/get.go:1293 +0x1e5
cmd/go/internal/modget.runGet({0xa82780, 0xc0000b8000}, 0xc0000e01e0, {0xc0000b2170, 0x1, 0x1})
        /usr/local/google/home/bcmills/go/src/cmd/go/internal/modget/get.go:300 +0x374
main.invoke(0xd9be40, {0xc0000b2160, 0x2, 0x2})
        /usr/local/google/home/bcmills/go/src/cmd/go/main.go:216 +0x2f6
main.main()
        /usr/local/google/home/bcmills/go/src/cmd/go/main.go:173 +0x78e

I'll investigate, and I should have a fix for either 1.17.1 or (possibly) 1.17.2.

@bcmills
Copy link
Contributor

bcmills commented Aug 30, 2021

Somehow go get is ending up thinking that it needs github.com/google/gofuzz@v1.1.0. Adding v1.2.0 explicitly causes the command to succeed:

~/tmp/gitlab-agent$ go1.17 get gitlab.com/gitlab-org/labkit@fc0c07abad6a32b5b77e16289fcc72d38f8411ab github.com/google/gofuzz@v1.2.0
go get: upgraded github.com/Microsoft/go-winio v0.4.19 => v0.5.0
go get: upgraded github.com/google/gofuzz v1.1.0 => v1.2.0
go get: upgraded github.com/lightstep/lightstep-tracer-go v0.24.0 => v0.25.0
go get: upgraded github.com/philhofer/fwd v1.0.0 => v1.1.1
go get: upgraded github.com/uber/jaeger-client-go v2.27.0+incompatible => v2.29.1+incompatible
go get: upgraded gitlab.com/gitlab-org/labkit v1.9.0 => v1.9.1-0.20210826103322-fc0c07abad6a
go get: upgraded gopkg.in/DataDog/dd-trace-go.v1 v1.30.0 => v1.32.0

There really doesn't seem to be any reason to reject github.com/google/gofuzz v1.2.0, so the panic here really does indicate a bogus constraint. Still investigating how that bogus constraint gets locked in.

@bcmills
Copy link
Contributor

bcmills commented Aug 30, 2021

Ah, I think I've got it!

gitlab.com/gitlab-org/labkit@fc0c07abad6a32b5b77e16289fcc72d38f8411ab specifies go1.17, so this interacts with module graph pruning.

We have an existing indirect dependency on gopkg.in/DataDog/dd-trace-go.v1:

~/tmp/gitlab-agent$ go mod why -m gopkg.in/DataDog/dd-trace-go.v1
# gopkg.in/DataDog/dd-trace-go.v1
gitlab.com/gitlab-org/cluster-integration/gitlab-agent/v14/internal/tool/tracing
gitlab.com/gitlab-org/labkit/tracing/impl
gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentracer

When we upgrade to fc0c07ab, we pull in an upgrade of dd-trace-go.v1 to v1.32.0, which is still on go 1.12. So upgrading labkit indirectly pulls in the new dependencies of dd-trace-go.v1, even though those dependencies are pruned out of the labkit dependency itself.

@ash2k
Copy link
Contributor Author

ash2k commented Aug 30, 2021

@bcmills

so this interacts with module graph pruning.

Yes, I was trying to see if module graph pruning helps to reduce the number of dependencies. Thanks for investigating this!

@gopherbot
Copy link

Change https://golang.org/cl/347290 mentions this issue: cmd/go/internal/modload: scan dependencies of root paths when raising version limits in editRequirements

@bcmills
Copy link
Contributor

bcmills commented Sep 2, 2021

@gopherbot, please backport to Go 1.17. This is a cmd/go panic due to an internal invariant violation, which may be confusing for end users to diagnose.

@gopherbot
Copy link

Backport issue(s) opened: #48156 (for 1.17).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 2, 2021
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 2, 2021
@bcmills
Copy link
Contributor

bcmills commented Sep 2, 2021

I have a fix, and I expect that it should land for Go 1.17.1. (I also verified that it fixes the crash reported by @thomaspeugeot.)

Thanks again for the detailed steps to reproduce the bug!

@thomaspeugeot
Copy link

:-) thks

@gopherbot
Copy link

Change https://golang.org/cl/348411 mentions this issue: [release-branch.go1.17] cmd/go/internal/modload: scan dependencies of root paths when raising version limits in editRequirements

gopherbot pushed a commit that referenced this issue Sep 8, 2021
… root paths when raising version limits in editRequirements

Updates #47979
Fixes #48156

Change-Id: I1d9d854cda1378e20c70e6c6789b77e42e467ca7
Reviewed-on: https://go-review.googlesource.com/c/go/+/347290
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 409434d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/348411
@thomaspeugeot
Copy link

Works fine with 1.17.1.

Thks a lot & congrats !

@fgm
Copy link

fgm commented Apr 12, 2022

A very similar issue is present again on 1.18, though:

$ go version
go version go1.18 darwin/amd64
$ go get github.com/golangci/golangci-lint@v1.45
panic: internal error: can't find reason for requirement on github.com/iancoleman/strcase@v0.0.0-20191112232945-16388991a334

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc00063d520?, 0xc0003d8858?}, {0xc0000cf2c0?, 0xc000ae22c8?}})
	/usr/local/Cellar/go/1.18/libexec/src/cmd/go/internal/modget/get.go:1760 +0x114
cmd/go/internal/modget.(*resolver).updateBuildList(0xc0005fd600, {0x16dbd58, 0xc0000aa000}, {0x0, 0x0, 0x0})
	/usr/local/Cellar/go/1.18/libexec/src/cmd/go/internal/modget/get.go:1765 +0x593
cmd/go/internal/modget.(*resolver).resolveQueries(0xc0005fd600, {0x16dbd58, 0xc0000aa000}, {0xc0000c0048, 0x1, 0x106ed6f?})
	/usr/local/Cellar/go/1.18/libexec/src/cmd/go/internal/modget/get.go:1243 +0x1e5
cmd/go/internal/modget.runGet({0x16dbd58, 0xc0000aa000}, 0xc0000d8498?, {0xc0000a41a0, 0x1, 0x1})
	/usr/local/Cellar/go/1.18/libexec/src/cmd/go/internal/modget/get.go:314 +0x40b
main.invoke(0x19bdac0, {0xc0000a4190, 0x2, 0x2})
	/usr/local/Cellar/go/1.18/libexec/src/cmd/go/main.go:218 +0x2ee
main.main()
	/usr/local/Cellar/go/1.18/libexec/src/cmd/go/main.go:175 +0x78e

It does not happen on an empty repo created just to reproduce the issue, but only on larger ones with many other dependencies.

@bcmills
Copy link
Contributor

bcmills commented Apr 12, 2022

@fgm, please file a new issue with concrete steps to reproduce the failure. (A standalone repo that reproduces it would be ideal, even if it isn't minimal.)

@fgm
Copy link

fgm commented Apr 14, 2022

@bcmills sorry, I spent a couple of hours trying to reproduce it without involving proprietary repos without success, so I worked around it by editing the go.mod manually then doing a go mod tidy, which allowed the update to pass.

I'm wondering if your 1.17 fix for that issue was really forward ported to 1.18 ?

@rsc rsc unassigned bcmills Jun 23, 2022
@krasi-georgiev
Copy link

I just tried 1.19 and it also has this issue, don't have steps to repro as it happens on a private repo

@jdemeyer
Copy link

jdemeyer commented Sep 9, 2022

Also got this shortly after upgrading to 1.19 (but never before, so it feels like a 1.19 regression).

I solved it by manually deleting all require blocks in go.mod except for the first one and then running go mod tidy.

@Richard1ybb
Copy link

FYI, I met an identical issue.

go version go1.18.4 darwin/arm64

@bcmills
Copy link
Contributor

bcmills commented Sep 28, 2022

@Richard1ybb, @jdemeyer: if you're observing these errors on a current Go release, please file a new issue with steps to reproduce it.

(It seems likely that there is still a bug here, but the code is complex enough that we're unlikely to find it by just reading through the code.)

@golang golang locked as resolved and limited conversation to collaborators Sep 28, 2022
@golang golang unlocked this conversation Nov 2, 2022
@golang golang locked as resolved and limited conversation to collaborators Nov 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
GoCommand cmd/go modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

9 participants