Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: internal compiler error: devirtualization failed (x/text, x/build, protobuf) #42279

Closed
zikaeroh opened this issue Oct 29, 2020 · 9 comments
Closed

cmd/compile: internal compiler error: devirtualization failed (x/text, x/build, protobuf) #42279

zikaeroh opened this issue Oct 29, 2020 · 9 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone
Go1.16

Comments

@zikaeroh
Copy link
Contributor

zikaeroh commented Oct 29, 2020 •
edited

What version of Go are you using (go version)?

$ go version
go version devel +5cc43c51c9 Thu Oct 29 19:06:32 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

On tip, yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/.cache/go-build"
GOENV="/home/jake/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jake/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jake/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jake/sdk/gotip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/zikaeroh/hortbot/hortbot2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build236306354=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version devel +5cc43c51c9 Thu Oct 29 19:06:32 2020 +0000 linux/amd64
GOROOT/bin/go tool compile -V: compile version devel +5cc43c51c9 Thu Oct 29 19:06:32 2020 +0000
uname -sr: Linux 5.9.1-zen2-1-zen
/usr/lib/libc.so.6: GNU C Library (GNU libc) release release version 2.32.
gdb --version: GNU gdb (GDB) 9.2

What did you do?

Ran gotip test in my project to check for performance changes from the new inlining/devirtualization changes.

I don't think I need to provide the exact code; the error message shows which files in my module cache are failing to be built.

I'm guessing a build dashboard will reveal this, I'm just probably too quick to test changes I'm excited to see...

What did you expect to see?

A build success.

What did you see instead?

An ICE:

# golang.org/x/text/encoding
../../../go/pkg/mod/golang.org/x/text@v0.3.3/encoding/encoding.go:293:5: internal compiler error: devirtualization failed: transform.t.(*Decoder).Transformer.Reset

goroutine 1 [running]:
runtime/debug.Stack(0xdaf700, 0xc00012a008, 0x0)
	/home/jake/sdk/gotip/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0xca427c, 0x1b, 0xc0004c95c0, 0x1, 0x1)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc0004b8080)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc0004b8080, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc0004b8080, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspectList(0xc0004b2480, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc0004b8880, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1075 +0x66
cmd/compile/internal/gc.inspect(0xc00048d680, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1076 +0x85
cmd/compile/internal/gc.inspectList(0xc00011d8c0, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc00046fb00, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00011d8d8, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc000192580)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0xcbbce0)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/home/jake/sdk/gotip/src/cmd/compile/main.go:52 +0xb1

# github.com/golang/protobuf/proto
../../../go/pkg/mod/github.com/golang/protobuf@v1.4.3/proto/deprecated.go:18:2: internal compiler error: devirtualization failed: xtd.(*impl.extensionTypeDescriptor).ExtensionDescriptor.HasDefault

goroutine 1 [running]:
runtime/debug.Stack(0xdaf700, 0xc00000e018, 0x0)
	/home/jake/sdk/gotip/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0xca427c, 0x1b, 0xc0007235e0, 0x1, 0x1)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000429e80)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000429e80, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000429e80, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspectList(0xc00000d8f0, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000429d80, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00000d950, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000429a00, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00000da58, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc0001542c0)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0xcbbce0)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/home/jake/sdk/gotip/src/cmd/compile/main.go:52 +0xb1

FAIL	github.com/hortbot/hortbot/internal/bot [build failed]
FAIL

cc @mdempsky
@zikaeroh zikaeroh changed the title cmd/compile: internal compiler error: devirtualization failed cmd/compile: internal compiler error: devirtualization failed (x/text, protobuf) Oct 29, 2020
@mdempsky mdempsky self-assigned this Oct 29, 2020
@mdempsky mdempsky added NeedsFix The path to resolution is known, but the work has not been done. release-blocker labels Oct 29, 2020
@mdempsky mdempsky added this to the Go1.16 milestone Oct 29, 2020
@mdempsky
Copy link
Member

Thanks, on it.

@gopherbot
Copy link

Change https://golang.org/cl/266359 mentions this issue: cmd/compile: gracefully fail when devirtualization fails

@zikaeroh
Copy link
Contributor Author

Confusingly, golang.org/x/text@v0.3.3/encoding/encoding.go at line 293 isn't even what the error reports, its an errors.New call with a string.

var ErrInvalidUTF8 = errors.New("encoding: invalid UTF-8")

@kevinburke1
Copy link

In case more test cases helps

$ go build ./...
# github.com/meterup/auth/vendor/github.com/golang/protobuf/proto
vendor/github.com/golang/protobuf/proto/deprecated.go:18:2: internal compiler error: devirtualization failed: xtd.(*impl.extensionTypeDescriptor).ExtensionDescriptor.HasDefault

goroutine 1 [running]:
runtime/debug.Stack(0x19adbe0, 0xc0000b8008, 0x0)
	/Users/kevin/go/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0x18a1cc4, 0x1b, 0xc0008df5e0, 0x1, 0x1)
	/Users/kevin/go/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000142200)
	/Users/kevin/go/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000142200, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000142200, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspectList(0xc00041a108, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000142100, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00041a168, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc00013fd80, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00041a270, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc000419340)
	/Users/kevin/go/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0x18b95d8)
	/Users/kevin/go/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/Users/kevin/go/src/cmd/compile/main.go:52 +0xb1

that's github.com/golang/protobuf v1.4.2

@mdempsky
Copy link
Member

Confusingly, golang.org/x/text@v0.3.3/encoding/encoding.go at line 293 isn't even what the error reports, its an errors.New call with a string.

Yeah, I noticed that too. I think that's an artifact of Fatalf using lineno, but devirtualizeCall being new and written in a style that avoids setting lineno. So Fatalf is misreporting against the last time we set lineno (which was probably the last function to be analyzed for inlining).

@mdempsky
Copy link
Member

Looks like both of the reported failure cases are because of a struct type having a promoted method from an embedded interface-typed field. So we can't fully devirtualize those calls, but we can still improve them somewhat.

gopherbot pushed a commit that referenced this issue Oct 29, 2020
@mdempsky
cmd/compile: gracefully fail when devirtualization fails
aa4f48b 
We should still be able to devirtualize here, but I need to understand
the AST better. While I'm doing that, at least switch to a graceful
failure case (i.e., skip the optimization and print a warning message)
to fix the x/text builders.

Updates #42279.

Change-Id: Ie2b0b701fccf590d0cabfead703fc2fa999072cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/266359
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/266300 mentions this issue: cmd/compile: fix devirtualization of promoted interface methods

@zikaeroh zikaeroh mentioned this issue Oct 29, 2020
Closed
@dmitshur dmitshur changed the title cmd/compile: internal compiler error: devirtualization failed (x/text, protobuf) cmd/compile: internal compiler error: devirtualization failed (x/text, x/build, protobuf) Oct 30, 2020
@dmitshur
Copy link
Contributor

Two more data points from x/build:


# golang.org/x/build/cmd/coordinator [golang.org/x/build/cmd/coordinator.test]
cmd/coordinator/builders.go:43:5: internal compiler error: devirtualization failed: tlsLn.(*tls.listener).Listener.Addr

goroutine 1 [running]:
runtime/debug.Stack(0x19adbc0, 0xc0000ba008, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0x18a1cc4, 0x1b, 0xc0008f3618, 0x1, 0x1)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000077d80)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000077d80, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000077d80, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspect(0xc0014b9800, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1076 +0x85
cmd/compile/internal/gc.inspectList(0xc0000afda0, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000077c00, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc0000afe30, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc0001242c0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0x18b95d0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/main.go:52 +0xb1

# golang.org/x/build/devapp [golang.org/x/build/devapp.test]
devapp/data.go:12:2: internal compiler error: devirtualization failed: tlsLn.(*tls.listener).Listener.Addr

goroutine 1 [running]:
runtime/debug.Stack(0x19adbc0, 0xc0000c0008, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0x18a1cc4, 0x1b, 0xc000a29618, 0x1, 0x1)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000399180)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000399180, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000399180, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspect(0xc000efcf00, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1076 +0x85
cmd/compile/internal/gc.inspectList(0xc00000ced0, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000399000, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00000cf18, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc00042d4a0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0x18b95d0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/main.go:52 +0xb1

From https://build.golang.org/?repo=golang.org%2fx%2fbuild for Go commit 5cc43c5. (Other Go commits look okay.)

@mdempsky
Copy link
Member

@dmitshur Thanks. Those are both instances of promoted interface methods too, and should be handled correctly by e62adb1.

@golang golang locked and limited conversation to collaborators Oct 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Milestone
Go1.16
Development

No branches or pull requests
5 participants
@mdempsky @dmitshur @gopherbot @zikaeroh @kevinburke1