@aclements, please decide if this is happening now or kick it down the road and update its milestone.

Too invasive for the freeze. Commencing kicking.

I looked into this for an hour or so. The mechanism does not seem quite robust enough still. After arranging for GOTRACEBACK=all to pass a SIGQUIT around like in GOTRACEBACK=crash, I only managed to find the missing goroutine running on another thread about half the time. I assume the other half of the time it had stopped by the time the SIGQUIT came in. We will probably need to be more aggressive about searching for the missing goroutines in order to be sure to print them all. And doing so will require cleaning up the SIGQUIT token passing a bit, so I'm not posting my code here. It was awful.

Test program:

package main

import "time"

func main() {
    for i := 0; i < 3; i++ {
        go func() {
            select {}
        }()
    }
    go func() {
        for {
        }
    }()
    time.Sleep(2 * time.Millisecond)
    panic(1)
}

Goroutine 7 is the one that only shows up half the time.

Now that we have #24543 (asynchronous preemption) we could make it so that GOTRACEBACK=crash is the default, and add a check in the signal handler if we're trying to freeze-the-world, and let asynchronous preemption preempt even at an unsafe-point in that case. This would mean that we don't need the existing SIGQUIT-bouncing mechanism and the tracebacks would all appear together (instead of separated by SIGQUITs and register dumps).

Also, just to reference what restarted this conversation: census-instrumentation/opencensus-go#1200 was missing a goroutine in the default traceback, and folks had to add GOTRACEBACK=crash.

It should be noted that "show other threads" is not strictly "Y" for crash. It is "Y" for signal-generated panics (SIGSEGV, SIGABRT, etc), but "N" for standard panic/throw calls. This is because the SIGQUIT-bouncing mechanism is performed by the signal handler, which is bypassed by standard panic/throw.

i.e., for panic/throw, we raise SIGABRT with handlingSig[SIGABRT] = 0, and sigfwdgo immediately clears the sigaction and re-raises SIGABRT to immediately abort before sighandler can do the SIGQUIT dance.