Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile, x/sync/errgroup: TestWithContext failing after devirtualization CL #42284

Closed
zikaeroh opened this issue Oct 29, 2020 · 15 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone

Comments

@zikaeroh
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version devel +fe70a3a0fd Thu Oct 29 21:46:54 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/.cache/go-build"
GOENV="/home/jake/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jake/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jake/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jake/sdk/gotip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/zikaeroh/sync/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build875916662=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version devel +fe70a3a0fd Thu Oct 29 21:46:54 2020 +0000 linux/amd64
GOROOT/bin/go tool compile -V: compile version devel +fe70a3a0fd Thu Oct 29 21:46:54 2020 +0000
uname -sr: Linux 5.9.1-zen2-1-zen
/usr/lib/libc.so.6: GNU C Library (GNU libc) release release version 2.32.
gdb --version: GNU gdb (GDB) 9.2

What did you do?

In the golang.org/x/sync/errgroup package, run go test.

What did you expect to see?

Passing tests.

What did you see instead?

--- FAIL: TestWithContext (0.00s)
panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx [recovered]
	panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

goroutine 29 [running]:
testing.tRunner.func1.2(0x5b0e80, 0xc0000a4e70)
	/home/jake/sdk/gotip/src/testing/testing.go:1123 +0x332
testing.tRunner.func1(0xc000083080)
	/home/jake/sdk/gotip/src/testing/testing.go:1126 +0x4b6
panic(0x5b0e80, 0xc0000a4e70)
	/home/jake/sdk/gotip/src/runtime/panic.go:965 +0x1b9
golang.org/x/sync/errgroup_test.TestWithContext(0xc000083080)
	/home/jake/zikaeroh/sync/errgroup/errgroup_test.go:166 +0x652
testing.tRunner(0xc000083080, 0x5f4c88)
	/home/jake/sdk/gotip/src/testing/testing.go:1173 +0xef
created by testing.(*T).Run
	/home/jake/sdk/gotip/src/testing/testing.go:1218 +0x2b3
exit status 2
FAIL	golang.org/x/sync/errgroup	0.005s

https://build.golang.org/log/8759ffb3dd087325958f6d2d67773f3dd2f70cf2
https://build.golang.org/log/32b25547fddc3b46c217c9efffb9aa124df3766e

The dashboard shows this test consistently failing after the devirtualization CL, but this differs from #42279 as it affects runtime behavior.

cc @mdempsky

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 30, 2020
@dmitshur dmitshur added this to the Backlog milestone Oct 30, 2020
@dmitshur
Copy link
Contributor

Now that #42279 is resolved, is this still an issue? I'm not sure I understand how this issue is different.

@zikaeroh
Copy link
Contributor Author

See https://build.golang.org/?repo=golang.org%2fx%2fsync; the page is all red on tip.

@dmitshur
Copy link
Contributor

dmitshur commented Oct 30, 2020

Thanks. I can reproduce locally:


$ gotip version
go version devel +f7e26467b Fri Oct 30 01:31:10 2020 +0000 darwin/amd64
$ gotip test golang.org/x/sync/errgroup
--- FAIL: TestWithContext (0.00s)
panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx [recovered]
	panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

goroutine 14 [running]:
testing.tRunner.func1.2(0x11eab00, 0xc00007ae70)
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1123 +0x332
testing.tRunner.func1(0xc000154000)
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1126 +0x4b6
panic(0x11eab00, 0xc00007ae70)
	/Users/dmitshur/sdk/gotip/src/runtime/panic.go:965 +0x1b9
golang.org/x/sync/errgroup_test.TestWithContext(0xc000154000)
	/Users/dmitshur/go/src/golang.org/x/sync/errgroup/errgroup_test.go:166 +0x652
testing.tRunner(0xc000154000, 0x1233b40)
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1173 +0xef
created by testing.(*T).Run
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1218 +0x2b3
FAIL	golang.org/x/sync/errgroup	0.323s
FAIL

@dmitshur dmitshur modified the milestones: Backlog, Go1.16 Oct 30, 2020
@gopherbot
Copy link

Change https://golang.org/cl/266518 mentions this issue: cmd/compile: mark inlined functions's results as reassigned

@mdempsky mdempsky self-assigned this Oct 30, 2020
@mdempsky
Copy link
Member

Sorry, missed this yesterday. Investigating.

@mdempsky
Copy link
Member

A workaround for the issue would be to to modify errgroup.WithContext like this:

 func WithContext(ctx context.Context) (*Group, context.Context) {
-       ctx, cancel := context.WithCancel(ctx)
-       return &Group{cancel: cancel}, ctx
+       ctx1, cancel := context.WithCancel(ctx)
+       return &Group{cancel: cancel}, ctx1
 }

It's a compiler bug that the current code (i.e., reassigning ctx) is miscompiled, but this change would be desirable anyway because it would play better with the current devirtualization code.

@zikaeroh
Copy link
Contributor Author

It's a compiler bug that the current code (i.e., reassigning ctx) is miscompiled, but this change would be desirable anyway because it would play better with the current devirtualization code.

Reassigning over ctx is probably the most common thing to do with contexts; it'd be a shame if things didn't work properly without making variable name changes. But, maybe in fixing this bug it'll fix that too.

@gopherbot
Copy link

Change https://golang.org/cl/266618 mentions this issue: cmd/compile: fix reassignVisitor

@mdempsky
Copy link
Member

Reassigning over ctx is probably the most common thing to do with contexts; it'd be a shame if things didn't work properly without making variable name changes.

Once we move escape analysis to SSA, this should be straightforward to handle. But right now, the Node AST doesn't make it easy to do. We'd have to effectively reimplement SSA renaming in the frontend.

@zikaeroh
Copy link
Contributor Author

Ah, I had no idea this was at the AST level where you couldn't distinguish it. Clearly I'm not paying enough attention during your streams. 🙂

@cespare
Copy link
Contributor

cespare commented Oct 30, 2020

@mdempsky I'm really excited that we're getting these devirtualization optimizations and there are several places in my own code where I anticipate them making a big difference. That said, it's a little concerning when an optimization applies only when code is written in a non-standard (and arguably worse) way. Depending on how many release cycles the devirtualization handles the ctx1 case but not the ctx case, it's the kind of thing that can work its way into the performance lore. I really hope in a year or two I don't start seeing high-performance code has been manually SSA-ified to satisfy the devirtualizer :)

@mdempsky
Copy link
Member

@cespare That's a fair point. There's certainly a trade off though: the more effort put into improving the early devirtualization code is effort taken away from moving escape analysis to SSA. The current early devirtualization pass is basically just a minor extension of the inlining / escape analysis improvements implemented for #41474

If you do see that practice developing, let us know and we can revisit whether there are more cases that we can handle without too much trouble.

@kivikakk
Copy link

I might be off base here, disregard if so, but this ticket is the one search result for

panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

and I'm hitting it, so thought it may be related. It's unfortunately deeply nested in use (websocket library used by a Discord client library), I'm not quite able to extract a simple repro yet, and I'm on ARM64. Not a support request, just dropping this in case it's useful.

Here's the line that it hits on:

		case <-readCtx.Done():

I've tried to extract this into a self-contained reproduction, but it doesn't fail :( Here's what it looks like running:

$ ~/Code/gopath/bin/gotip version
go version devel +55b5801 Sat Dec 19 00:20:38 2020 +0000 darwin/arm64
$ ~/Code/gopath/bin/gotip run main.go 
panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

goroutine 66 [running]:
nhooyr.io/websocket.(*Conn).timeoutLoop(0x140000141a0)
        /Users/kameliya/Code/gopath/pkg/mod/nhooyr.io/websocket@v1.8.6/conn_notjs.go:161 +0x3d0
created by nhooyr.io/websocket.newConn
        /Users/kameliya/Code/gopath/pkg/mod/nhooyr.io/websocket@v1.8.6/conn_notjs.go:114 +0x37c
exit status 2

@mdempsky
Copy link
Member

@kivikakk Thanks for the report! I was able to create a reproducer.

@kivikakk
Copy link

Nice, thank you! I'll watch with interest.

@golang golang locked and limited conversation to collaborators Dec 20, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Development

No branches or pull requests

6 participants