Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: M woken with no P on Windows #35391

Closed
jazzy-crane opened this issue Nov 6, 2019 · 10 comments
Closed

runtime: M woken with no P on Windows #35391

jazzy-crane opened this issue Nov 6, 2019 · 10 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Milestone
Go1.19

Comments

@jazzy-crane
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.12.12 windows/amd64

Does this issue reproduce with the latest release?

Not a reproducible scenario unfortunately

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\ContainerAdministrator\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\gopath
set GOPROXY=
set GORACE=
set GOROOT=c:\go
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-LC:/winsdklibs64
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\ContainerAdministrator\AppData\Local\Temp\go-build221886546=/tmp/go-build -gno-record-gcc-switches

What did you do?

This was a one-off crash seen on our moderately large windows application written in Go. It occurred within a second of startup of the application - not immediately, many of our modules/packages successfully initialised.

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0x40 pc=0x43dae3]

runtime stack:
runtime.throw(0x1579f3c, 0x2a)
	c:/go/src/runtime/panic.go:617 +0x79
runtime.sigpanic()
	c:/go/src/runtime/signal_windows.go:227 +0x272
runtime.wirep(0x0)
	c:/go/src/runtime/proc.go:4111 +0x43
runtime.acquirep(0x0)
	c:/go/src/runtime/proc.go:4086 +0x32
runtime.stopm()
	c:/go/src/runtime/proc.go:1938 +0xf9
runtime.startlockedm(0xc000314600)
	c:/go/src/runtime/proc.go:2106 +0x91
runtime.schedule()
	c:/go/src/runtime/proc.go:2555 +0x77
runtime.park_m(0xc0005e4480)
	c:/go/src/runtime/proc.go:2605 +0xb6
runtime.mcall(0x2000)
	c:/go/src/runtime/asm_amd64.s:299 +0x5e

Apologies that I can't provide much information to narrow this down. I'm hoping the stacktrace alone is interesting - it looks to me that runtime.wirep shouldn't be called with a nil *p pointer. I suppose I can't rule out heap corruption, but we're not in the midst of seeing lots of one-off crashes in development.

What did you expect to see?

No runtime crash - application completes initialisation

What did you see instead?

Runtime crash midway through application initialisation
@ianlancetaylor
Copy link
Member

Thanks for the report. Unfortunately I just looked through every code path and I don't see how this is possible. My best guess is that somehow WaitForSingleObject returned before it was signaled. Or I suppose it is conceivable that there was no memory fence between the call to SetEvent and the return from WaitFromSingleObject, although that seems implausible.

This code has changed slightly on tip, but not in a way that I would expect to make a difference.

I guess we will keep this issue open and see if it happens again.

@ianlancetaylor ianlancetaylor changed the title "fatal error: unexpected signal during runtime execution" on windows/amd64 runtime: M woken with no P on Windows Nov 6, 2019
@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows labels Nov 6, 2019
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Nov 6, 2019
@egonelbre egonelbre mentioned this issue Oct 14, 2020
Closed
@adriansr
Copy link

adriansr commented Mar 2, 2022

Observing this issue frequently on a program built with Go 1.16.6 running under Windows Server 2016:

fatal error: unexpected signal during runtime execution
fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0x38 pc=0xabad15]

runtime stack:
runtime.throw(0x45b1379, 0x2a)
        /usr/local/go/src/runtime/panic.go:1117 +0x79
runtime.sigpanic()
        /usr/local/go/src/runtime/signal_windows.go:233 +0x317
runtime.wirep(0x0)
        /usr/local/go/src/runtime/proc.go:4986 +0x35
runtime.acquirep(0x0)
        /usr/local/go/src/runtime/proc.go:4961 +0x32
runtime.stopm()
        /usr/local/go/src/runtime/proc.go:2302 +0xbe
runtime.findrunnable(0xc00005c000, 0x0)
        /usr/local/go/src/runtime/proc.go:2960 +0x74e
runtime.schedule()
        /usr/local/go/src/runtime/proc.go:3169 +0x2d7
runtime.park_m(0xc000107200)
        /usr/local/go/src/runtime/proc.go:3318 +0xb2
runtime.mcall(0x2000)
        /usr/local/go/src/runtime/asm_amd64.s:327 +0x5e

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0x38 pc=0x39ad15]

runtime stack:
runtime.throw(0x3e91379, 0x2a)
        /usr/local/go/src/runtime/panic.go:1117 +0x79
runtime.sigpanic()
        /usr/local/go/src/runtime/signal_windows.go:233 +0x317
runtime.wirep(0x0)
        /usr/local/go/src/runtime/proc.go:4986 +0x35
runtime.acquirep(0x0)
        /usr/local/go/src/runtime/proc.go:4961 +0x32
runtime.stopm()
        /usr/local/go/src/runtime/proc.go:2302 +0xbe
runtime.findrunnable(0xc00005e800, 0x0)
        /usr/local/go/src/runtime/proc.go:2960 +0x74e
runtime.schedule()
        /usr/local/go/src/runtime/proc.go:3169 +0x2d7
runtime.park_m(0xc000b63680)
        /usr/local/go/src/runtime/proc.go:3318 +0xb2
runtime.mcall(0x2000)
        /usr/local/go/src/runtime/asm_amd64.s:327 +0x5e

/cc @ianlancetaylor

@ianlancetaylor
Copy link
Member

CC @mknyszek @prattmic @aclements

@ianlancetaylor ianlancetaylor modified the milestones: Unplanned, Go1.19 Mar 3, 2022
@mknyszek
Copy link
Contributor

mknyszek commented Mar 3, 2022

@adriansr Since you're getting this frequently, does it reproduce with the most recent releases? (Go 1.17 and/or Go 1.18rc1.)

@adriansr
Copy link

adriansr commented Mar 3, 2022 •
edited
Loading

Yes, just tested with Go 1.17.6:

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0x38 pc=0x9670e6]

runtime stack:
runtime.throw({0x416e9cf, 0x96963d})
        /usr/local/go/src/runtime/panic.go:1198 +0x76
runtime.sigpanic()
        /usr/local/go/src/runtime/signal_windows.go:248 +0x22b
runtime.wirep(0xc000f851e0)
        /usr/local/go/src/runtime/proc.go:5160 +0x26
runtime.acquirep(0x0)
        /usr/local/go/src/runtime/proc.go:5135 +0x1e
runtime.stopm()
        /usr/local/go/src/runtime/proc.go:2409 +0x8d
runtime.findrunnable()
        /usr/local/go/src/runtime/proc.go:2984 +0x885
runtime.schedule()
        /usr/local/go/src/runtime/proc.go:3367 +0x239
runtime.park_m(0xc000e6f1e0)
        /usr/local/go/src/runtime/proc.go:3516 +0x14d
runtime.mcall()
        /usr/local/go/src/runtime/asm_amd64.s:307 +0x4a

I'm wondering why in this case the _p_ *p parameter for wirep() is not displayed as nil, but the addr corresponds to _p_ being nil for _p_.m.

Will try to test with 1.18rc1 too.

@adriansr
Copy link

adriansr commented Mar 3, 2022

Same with 1.18rc1

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0x38 pc=0x1205fa6]

runtime stack:
runtime.throw({0x44b3033?, 0x490?})
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/panic.go:992 +0x76
runtime.sigpanic()
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/signal_windows.go:249 +0x23b
runtime.wirep(0xc00084e680?)
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/proc.go:4889 +0x26
runtime.acquirep(0x0)
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/proc.go:4864 +0x1e
runtime.stopm()
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/proc.go:2229 +0xb6
runtime.findrunnable()
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/proc.go:2804 +0x885
runtime.schedule()
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/proc.go:3187 +0x239
runtime.park_m(0xc00084e340?)
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/proc.go:3336 +0x14d
runtime.mcall()
	/Users/adrian/.gvm/versions/go1.18rc1.darwin.amd64/src/runtime/asm_amd64.s:425 +0x4a

I'll try to figure out what our app is doing at the moment of the error. This is Metricbeat from https://github.com/elastic/beats/ using a huge configuration from the user who reported the issue.

@prattmic
Copy link
Member

prattmic commented Mar 3, 2022

Since we just saw an issue with the beats seccomp policy (#51315), could you verify that this reproduces with the seccomp policy disabled? Likely unrelated, but it would help to eliminate that possibility.

@adriansr adriansr mentioned this issue Mar 4, 2022
Closed
@adriansr
Copy link

adriansr commented Mar 4, 2022

Thanks @prattmic. We don't have a seccomp policy on Windows, but indeed this was our fault due to bad lifetime management of Event objects passed to Windows Performance Counter APIs.

I left a write up in elastic/beats#30686 in case someone faces a similar issue.

@aclements
Copy link
Member

@adriansr , to clarify, are you saying this was caused by user code, in which case we can go ahead and close this issue? Thanks.

@adriansr
Copy link

adriansr commented Mar 7, 2022

@aclements , exactly, fault in user code.

@golang golang locked and limited conversation to collaborators Mar 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Projects
None yet
Milestone
Go1.19
Development

No branches or pull requests
7 participants
@mknyszek @prattmic @aclements @ianlancetaylor @gopherbot @adriansr @jazzy-crane