Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: unexpected signal during async preemption on windows/amd64 #44679

Closed
zx2c4 opened this issue Feb 28, 2021 · 10 comments
Closed

runtime: unexpected signal during async preemption on windows/amd64 #44679

zx2c4 opened this issue Feb 28, 2021 · 10 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Milestone
Backlog

Comments

@zx2c4
Copy link
Contributor

zx2c4 commented Feb 28, 2021

With @rsc's arm64 changes, I'm getting reports of crashes on amd64 involving signals during async preemption:

2021-02-28 10:31:31.978278: [TUN] fatal error: unexpected signal during runtime execution
2021-02-28 10:31:31.981817: [TUN] [signal 0xc0000005 code=0x0 addr=0xb1 pc=0xaf626a]
2021-02-28 10:31:31.981817: [TUN] runtime stack:
2021-02-28 10:31:31.981817: [TUN] runtime.throw(0xeffc67, 0x2a)
2021-02-28 10:31:31.981817: [TUN] runtime/panic.go:1117 +0x79
2021-02-28 10:31:31.981817: [TUN] runtime.sigpanic()
2021-02-28 10:31:31.981817: [TUN] runtime/signal_windows.go:237 +0x317
2021-02-28 10:31:31.981817: [TUN] runtime.wantAsyncPreempt(...)
2021-02-28 10:31:31.981817: [TUN] runtime/preempt.go:344
2021-02-28 10:31:31.981817: [TUN] runtime.preemptM(0xc000035400)
2021-02-28 10:31:31.981817: [TUN] runtime/os_windows.go:1306 +0x24a
2021-02-28 10:31:31.981817: [TUN] runtime.preemptone(0xc000027800, 0xf9061ffdc8)
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:5307 +0xb2
2021-02-28 10:31:31.981817: [TUN] runtime.retake(0x2de1c0936154, 0x200000000)
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:5214 +0x245
2021-02-28 10:31:31.981817: [TUN] runtime.sysmon()
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:5156 +0x196
2021-02-28 10:31:31.981817: [TUN] runtime.mstart1()
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:1317 +0xda
2021-02-28 10:31:31.981817: [TUN] runtime.mstart0()
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:1275 +0x6d
2021-02-28 10:31:31.981817: [TUN] runtime.mstart(0x7ffe62037034, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffe634fd241, 0x0, 0x0, 0x0, ...)
2021-02-28 10:31:31.981817: [TUN] runtime/asm_amd64.s:256 +0x5
2021-02-28 10:31:31.981817: [TUN] goroutine 1 [syscall, locked to thread]:
2021-02-28 10:31:31.981817: [TUN] syscall.Syscall(0x7ffe615c7c00, 0x1, 0xc00010fc58, 0x0, 0x0, 0x0, 0x0, 0x0)
2021-02-28 10:31:31.981817: [TUN] runtime/syscall_windows.go:332 +0xe9
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys/windows.StartServiceCtrlDispatcher(0xc00010fc58, 0xf1e5a8, 0xc000043540)
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys@v0.0.0-20210225134936-a50acf3fe073/windows/zsyscall_windows.go:1154 +0x8a
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys/windows/svc.Run(0xc000012258, 0x18, 0xf7cf00, 0xc000037cf0, 0x8, 0xc000012258)
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys@v0.0.0-20210225134936-a50acf3fe073/windows/svc/service.go:366 +0x1c8

Cc @aclements who wrote preemption support.
@zx2c4
Copy link
Contributor Author

zx2c4 commented Feb 28, 2021

Looking more carefully,

func wantAsyncPreempt(gp *g) bool {
	// Check both the G and the P.
	return (gp.preempt || gp.m.p != 0 && gp.m.p.ptr().preempt) && readgstatus(gp)&^_Gscan == _Grunning
}

Is this panicking because gp.m.p.ptr() points somewhere bogus?

@zx2c4
Copy link
Contributor Author

zx2c4 commented Feb 28, 2021

Ahhh more likely is that gp is bogus. That comes from:

	// Does it want a preemption and is it safe to preempt?
	gp := gFromSP(mp, c.sp())
	if wantAsyncPreempt(gp) {

@zx2c4
Copy link
Contributor Author

zx2c4 commented Feb 28, 2021

@rsc might be an issue with ece954d#diff-fc175f04ebb256c1d34c14d27b8915f38928b71df55a35bfbd86fcb4618ff5a9

Maybe it's simply enough to check for nil... I'll send a CL.

@gopherbot
Copy link

Change https://golang.org/cl/297390 mentions this issue: runtime: see whether gp==nil before checking preemption state

@zx2c4
Copy link
Contributor Author

zx2c4 commented Feb 28, 2021

I've deployed https://golang.org/cl/297390 and it evidently fixes the crash users were seeing.

@networkimprov
Copy link

@gopherbot add OS-Windows

@dmitshur dmitshur added this to the Backlog milestone Mar 2, 2021
@dmitshur dmitshur changed the title windows/amd64: unexpected signal during async preemption runtime: unexpected signal during async preemption on windows/amd64 Mar 2, 2021
@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. arch-arm64 and removed arch-arm64 labels Mar 2, 2021
@bcmills
Copy link
Contributor

bcmills commented Mar 17, 2021

Is https://build.golang.org/log/995be8dc15fe8289ca78257701e522e5b5d51336 (windows-386) likely to be due to the same root cause?

@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2021

Another windows-386 TryBot failure with unexpected signal during runtime execution in runtime.wantAsyncPreempt:
https://storage.googleapis.com/go-build-log/764770a7/windows-386-2008_4820c228.log

@zx2c4
Copy link
Contributor Author

zx2c4 commented Apr 6, 2021

Looks familiar. https://golang.org/cl/297390 fixes the issue, but I'm not able to reproduce it myself to get more info supporting that CL.

@zx2c4
Copy link
Contributor Author

zx2c4 commented Apr 6, 2021

CC @cherrymui

@golang golang locked and limited conversation to collaborators Apr 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@zx2c4 @networkimprov @dmitshur @bcmills @gopherbot