-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime/pprof: frequent test timeouts on builders since mid-August #41014
Comments
2020-08-24T20:22:16-5c76382/linux-386-sid Given the rash of failures in August, this is looking like a 1.16 regression, so marking as release-blocker. |
https://storage.googleapis.com/go-build-log/8921b4e8/android-amd64-emu_69e10eb0.log (a TryBot run) @golang/osp-team, could we bisect and revert the CL that introduced this apparent regression? |
Output from @aclements'
None of those “likely culprits” seem all that likely to me given the failure mode. (Perhaps it's more subtle than I expect, or perhaps the actual culprit didn't make the list.) |
Interestingly, the stack trace looks so broken. :-(
|
These two recent cases have an interesting stack trace: 2020-09-02T23:39:37-0e48c67/linux-arm64-packet
They are trying to acquire |
OK, how about this.
In that case the signal handler will call There is a comment in // Stop profiler on this thread so that it is safe to lock prof.
// if a profiling signal came in while we had prof locked,
// it would deadlock. but I don't see anything there that actually stops the profiler on this thread. So I think we have the very deadlock that that comment warns about. Perhaps it used to work but somehow changed (in which case it is probably my fault). |
Yes, this was my fault. Caused by https://golang.org/cl/240003. |
Change https://golang.org/cl/253758 mentions this issue: |
Thanks for tracking it down! |
@ianlancetaylor Sorry for replying on an issue that has been closed for a long time. Here I don't understand why setcpuprofilerate will never return, it only calls setProcessCPUProfiler(0) and then setitimer(_ITIMER_PROF, &itimerval{}, nil) before releasing prof.signalLock, will these two functions block? |
@erifan The problem would occur if a |
Oh I got it, thanks @ianlancetaylor |
2020-08-22T03:46:17-13e41bc/android-386-emu
2020-08-19T03:49:06-bd519d0/android-386-emu
Also https://storage.googleapis.com/go-build-log/1be3e58d/openbsd-amd64-64_e5acea9b.log.
CC @hyangah
The text was updated successfully, but these errors were encountered: