-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime/pprof: TestMutexProfile flakiness #19139
Comments
That's a remarkably uninformative error message, even if I wrote it. It
should include "expected to see evidence of a contended mutex, but didn't"
I'll try to figure out when this started and why.
…On Fri, Feb 17, 2017 at 1:43 AM, Keith Randall ***@***.***> wrote:
On the builders, I see TestMutexProfile occasionally failing with
--- FAIL: TestMutexProfile (0.01s)
pprof_test.go:532: expected 6 lines, got 3 "--- mutex:\ncycles/second=2362842166 <(236)%20284-2166>\nsampling period=1"
--- mutex:
cycles/second=2362842166 <(236)%20284-2166>
sampling period=1
FAIL
FAIL runtime/pprof 1.298s
Maybe freebsd only?
@pjweinb <https://github.com/pjweinb>
Some builder logs:
https://build.golang.org/log/c625cca078dac40bc5f1f50b62891c04b52308c5
https://build.golang.org/log/118de984fc0903fe93c3e677962cd7965637a2a3
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#19139>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AFKAIztybRHcmJ3GgqtJvSV8-C2pLYPkks5rdUF8gaJpZM4MD7Xl>
.
|
I also saw it on OpenBSD in #19146 (closed as dup of this). |
Is there some way of scanning the builder logs to find all the occurrences?
…On Fri, Feb 17, 2017 at 11:39 AM, Brad Fitzpatrick ***@***.*** > wrote:
I also saw it on OpenBSD in #19146
<#19146> (closed as dup of this).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#19139 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFKAI3_JmOy1T8ejJ2UVoDbIjfSZW1DAks5rdc1EgaJpZM4MD7Xl>
.
|
@aclements has a tool that scans the builder logs for test failures. |
|
Thank you. one suspects 58d7621 runtime: run mutexevent profiling without
holding semaRoot lock
.
…On Fri, Feb 17, 2017 at 12:39 PM, Austin Clements ***@***.***> wrote:
$ greplogs -dashboard -E "TestMutexProfile" -E "expected 6 lines" -l -md
2017-02-16T17:34:39-fc456c7/freebsd-386-gce101
<https://build.golang.org/log/650f030384a8cd59590aa45a7ba4b45162ecf13c>
2017-02-16T17:34:39-fc456c7/freebsd-amd64-race
<https://build.golang.org/log/acda774eebe4d038055f9b18ef8b5fa7e65c2b8b>
2017-02-16T17:52:15-990124d/linux-386
<https://build.golang.org/log/c85606a6439012033aea7647cf65e7dfb99f565d>
2017-02-16T21:54:59-661e217/freebsd-amd64-gce93
<https://build.golang.org/log/118de984fc0903fe93c3e677962cd7965637a2a3>
2017-02-16T21:54:59-661e217/linux-ppc64le-buildlet
<https://build.golang.org/log/c5f1775d3d40fb1dd1d050b0f3cf6213d93510c3>
2017-02-16T22:42:36-794f1eb/freebsd-amd64-race
<https://build.golang.org/log/80a6635a54baaea90fac0f91d3aafa17b068e1b3>
2017-02-17T06:16:44-708ba22/freebsd-amd64-gce93
<https://build.golang.org/log/c625cca078dac40bc5f1f50b62891c04b52308c5>
2017-02-17T14:36:27-79f6a5c/windows-386-gce
<https://build.golang.org/log/4ef0671c856abe449ef7aa8d2e5d75ef844c6e14>
$ greplogs -dashboard -E "TestMutexProfile" -E "expected 6 lines" -l | findflakes -paths
First observed fc456c7 16 Feb 17:34 2017 (8 commits ago)
Last observed 79f6a5c 17 Feb 14:36 2017 (1 commits ago)
29% chance failure is still happening
71% failure probability (6 of 8 commits)
Likely culprits:
71% fc456c7 cmd/compile/internal/gc: drop unused src.XPos params in SSA builder
20% 58d7621 runtime: run mutexevent profiling without holding semaRoot lock
No known past failures
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#19139 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFKAIwd02Uc7BzETFHTi9pRFA35m_XbJks5rddthgaJpZM4MD7Xl>
.
|
Seeing as it's also appearing on Linux and Windows, it appears OS agnostic. |
CL https://golang.org/cl/37291 mentions this issue. |
Flaky tests hurt productivity. Disable for now. Updates #19139 Change-Id: I2e3040bdf0e53597a1c4f925b788e3268ea284c1 Reviewed-on: https://go-review.googlesource.com/37291 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Peter Weinberger <pjw@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
@pjweinb, any progress on this? |
Never trust a commit that says "Clearly safe". :) I've confirmed that I can reproduce this fairly easily on master (738739f), but have been unable to reproduce it after locally reverting that change. |
Here's my hunch:
Some evidence that this is what's going on: If you add The good news is, if I'm right, this is purely a test bug. |
CL https://golang.org/cl/45751 mentions this issue. |
On the builders, I see TestMutexProfile occasionally failing with
Maybe freebsd only?
@pjweinb
Some builder logs:
https://build.golang.org/log/c625cca078dac40bc5f1f50b62891c04b52308c5
https://build.golang.org/log/118de984fc0903fe93c3e677962cd7965637a2a3
The text was updated successfully, but these errors were encountered: