Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: "signal 23 received but handler not on signal stack" after CL 485500 when signal delivered on shallow g0 stack by TSAN #60007

Closed
prattmic opened this issue May 5, 2023 · 2 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@prattmic
Copy link
Member

prattmic commented May 5, 2023

This bug is more-or-less the opposite of #59294.

On the first call into Go from a C thread, we get the precise g0 lower stack bound from pthread, but leave the upper bound imprecise, based on the SP at the point of the call (https://cs.opensource.google/go/go/+/master:src/runtime/proc.go;l=1958-1970;drc=265d19ed526b6d6a01a20150918b362c1e6befba;bpv=0;bpt=0).

After https://go.dev/cl/485500, since we keep using the same M we do not recompute the stack bounds on subsequent calls. If the first call has a deep SP and a subsequent call has a more shallow SP, may may be executing with SP > gp.stack.hi.

Typically this doesn't matter because we don't usually check mp.g0.stack.hi.

One place we do check mp.g0.stack.hi is when handling a signal delivered on g0 (https://cs.opensource.google/go/go/+/master:src/runtime/signal_unix.go;l=558-573;drc=72c33a5ef0eea8663328375d9d339ed150310ebb). This only happens when using TSAN. Normal signal delivery uses the sigaltstack mp.gsignal.stack, however TSAN queues signals and then delivers them from the normal thread stack (i.e., mp.g0.stack).

Thus the complete sequence is:

  1. C thread calls into Go with deep SP SP1.
  2. We set g0.stack.hi = SP1 + 1024.
  3. TSAN deliveries a signal with shallow SP SP2.
  4. adjustSignalStack sees SP2 > g0.stack.hi and determine that we aren't on the g0 stack, and throws with something like.
signal 23 received but handler not on signal stack
mp.gsignal 0x7fbad069e000 0x7fbad06ee000 mp.g0 0x7fbad0986000 0x7fbad09d4820 sp 0x7fbad09d4f98
fatal error: non-Go code set up signal handler without SA_ONSTACK flag

@cherrymui has a prototype fix to fetch the precise upper bound from pthread as well, which fixes this issue.

cc @cherrymui @doujiang24

@prattmic prattmic added the NeedsFix The path to resolution is known, but the work has not been done. label May 5, 2023
@prattmic prattmic added this to the Go1.21 milestone May 5, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label May 5, 2023
@gopherbot
Copy link

Change https://go.dev/cl/492987 mentions this issue: runtime: get better g0 stack high bound in needm

@gopherbot
Copy link

Change https://go.dev/cl/495855 mentions this issue: runtime/cgo: store M for C-created thread in pthread key

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

4 participants
@prattmic @gopherbot @cherrymui and others