Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: crash during generating CPU profile #66194

Open
secDre4mer opened this issue Mar 8, 2024 · 1 comment
Open

runtime: crash during generating CPU profile #66194

secDre4mer opened this issue Mar 8, 2024 · 1 comment
Assignees
@cherrymui
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@secDre4mer
Copy link
Contributor

secDre4mer commented Mar 8, 2024 •
edited

Go version

go version 1.20.13 windows/amd64

Output of go env in your module/workspace:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=".exe"
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="windows"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/3rdparty/_3rdparty/tgt/golang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/3rdparty/_3rdparty/tgt/golang/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.13"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="x86_64-w64-mingw32-gcc"
CXX="x86_64-w64-mingw32-g++"
CGO_ENABLED="1"
GOMOD="<redacted>"
GOWORK=""
CGO_CFLAGS="-I/3rdparty/_3rdparty/tgt/x86_64-w64-mingw32/include"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-L/3rdparty/_3rdparty/tgt/x86_64-w64-mingw32/lib"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build208241099=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Program ran for some time with CPU profile being generated in the background before crashing.
Unfortunately, could not reproduce the issue so far. Seen only once in a customer environment.

What did you see happen?

Program crashes with stack trace:

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0x150 pc=0xeef4f8]

runtime stack:
runtime.throw({0x25dd44c?, 0xee8?})
	/3rdparty/_3rdparty/tgt/golang/src/runtime/panic.go:1047 +0x65 fp=0x2fdfed20 sp=0x2fdfecf0 pc=0xecb1c5
runtime.sigpanic()
	/3rdparty/_3rdparty/tgt/golang/src/runtime/signal_windows.go:252 +0x265 fp=0x2fdfed68 sp=0x2fdfed20 pc=0xee0205
runtime.gentraceback(0x2fdff830?, 0x774a15d6?, 0x7ffc0000001?, 0xffffffffffffffff?, 0x2fdff8a0?, 0x0?, 0xc0003dc678?, 0x77482863?, 0xadae?, 0x6)
	/3rdparty/_3rdparty/tgt/golang/src/runtime/traceback.go:80 +0xf8 fp=0x2fdff0c8 sp=0x2fdfed68 pc=0xeef4f8
runtime.sigprof(0x0, 0x380?, 0x2fdff370?, 0x0, 0xc000080800)
	/3rdparty/_3rdparty/tgt/golang/src/runtime/proc.go:4750 +0x1b9 fp=0x2fdff338 sp=0x2fdff0c8 pc=0xed71f9
runtime.profilem(0xc000080800, 0x380?)
	/3rdparty/_3rdparty/tgt/golang/src/runtime/os_windows.go:1228 +0xfa fp=0x2fdff858 sp=0x2fdff338 pc=0xec7eda
runtime.profileLoop()
	/3rdparty/_3rdparty/tgt/golang/src/runtime/os_windows.go:1284 +0x1cc fp=0x2fdff8d0 sp=0x2fdff858 pc=0xec80ec
runtime.mstart1()
	/3rdparty/_3rdparty/tgt/golang/src/runtime/proc.go:1499 +0x9d fp=0x2fdff8f8 sp=0x2fdff8d0 pc=0xed041d
runtime.mstart0()
	/3rdparty/_3rdparty/tgt/golang/src/runtime/proc.go:1456 +0x6c fp=0x2fdff920 sp=0x2fdff8f8 pc=0xed034c
runtime.mstart()
	/3rdparty/_3rdparty/tgt/golang/src/runtime/asm_amd64.s:394 +0x5 fp=0x2fdff928 sp=0x2fdff920 pc=0xefcac5

Based on the stack trace, the runtime crashes while creating the CPU profile. More specifically, the gp used in sigprof seems to be nil. Based on the code, I suppose this can happen - sigprof explicitly has a comment that states this: Note: it can happen on Windows that we interrupted a system thread with no g, so gp could nil.

go1.22 still seems to be have the same issue from what I can tell (initAt assumes that gp is never nil, and sigprof never checks this).

What did you expect to see?

CPU profile should be collected without crash
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Mar 8, 2024
@mknyszek
Copy link
Contributor

mknyszek commented Mar 8, 2024

CC @golang/runtime

@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 8, 2024
@mknyszek mknyszek added this to the Backlog milestone Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cherrymui cherrymui

Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Go Compiler / Runtime
Status: Todo
Milestone
Backlog
Development

No branches or pull requests
4 participants
@mknyszek @gopherbot @cherrymui @secDre4mer