runtime: TestCgoPprof and TestCgoPprofPIE fail sometimes on linux/amd64 #18856
Comments
|
Copying data here for ease of reading: First: Second: Third: |
|
What version of GCC are you using? What does What do you see if you do this: |
|
@ianlancetaylor, GCC version is This is the top from profile I've attached to issue's description: Also, if I run it like |
|
Interesting. I never see any instances of |
|
Actually, with go1.7.5 tests pass just fine. I've collected some data using this script: go build && rm prof.txt &&
for i in {1..1000}; do
go tool pprof -nodecount 1 -top $(GOMAXPROCS=<N> ./testprogcgo CgoPprof) >> prof.txt;
done
So, with 3e55059 and higher GOMAXPROCS |
|
Yes, it looks like a regression, but I still can't recreate it and I still have no idea why it is happening for you. Can you find out if there are any calls to |
Maybe, I did something wrong, but after applying this patch, running diff --git a/src/runtime/proc.go b/src/runtime/proc.go
index f41672d..9f1859d 100644
--- a/src/runtime/proc.go
+++ b/src/runtime/proc.go
@@ -3379,7 +3379,9 @@ func sigprofNonGo() {
//go:nosplit
//go:nowritebarrierrec
func sigprofNonGoPC(pc uintptr) {
+ println("runtime.sigprofNonGoPC called")
if prof.hz != 0 {
+ println("runtime.sigprofNonGoPC \"hz != 0\" branch")
pc := []uintptr{
pc,
funcPC(_ExternalCode) + sys.PCQuantum,
@@ -3390,6 +3392,7 @@ func sigprofNonGoPC(pc uintptr) {
osyield()
}
if prof.hz != 0 {
+ println("runtime.sigprofNonGoPC \"cpuprof.addNonGo(pc)\" branch")
cpuprof.addNonGo(pc)
}
atomic.Store(&prof.lock, 0) |
|
@ianlancetaylor, I dug a bit and tried another patch. Here it is: diff --git a/src/runtime/proc.go b/src/runtime/proc.go
index f41672d..317fae7 100644
--- a/src/runtime/proc.go
+++ b/src/runtime/proc.go
@@ -3174,14 +3174,23 @@ func _System() { _System() }
func _ExternalCode() { _ExternalCode() }
func _GC() { _GC() }
+var (
+ Sigprof, SigprofHz0, SigprofPCMoreFMD bool
+ SigprofNonGo bool
+ SigprofNonGoPC, SigprofNonGoPCHz0, SigprofNonGoPCAddNonGo bool
+)
+
// Called if we receive a SIGPROF signal.
// Called by the signal handler, may run during STW.
//go:nowritebarrierrec
func sigprof(pc, sp, lr uintptr, gp *g, mp *m) {
+ Sigprof = true
if prof.hz == 0 {
return
}
+ SigprofHz0 = true
+
// Profiling runs concurrently with GC, so it must not allocate.
// Set a trap in case the code does allocate.
// Note that on windows, one thread takes profiles of all the
@@ -3317,6 +3326,7 @@ func sigprof(pc, sp, lr uintptr, gp *g, mp *m) {
n = 2
// "ExternalCode" is better than "etext".
if pc > firstmoduledata.etext {
+ SigprofPCMoreFMD = true
pc = funcPC(_ExternalCode) + sys.PCQuantum
}
stk[0] = pc
@@ -3354,6 +3364,7 @@ var sigprofCallersUse uint32
//go:nosplit
//go:nowritebarrierrec
func sigprofNonGo() {
+ SigprofNonGo = true
if prof.hz != 0 {
n := 0
for n < len(sigprofCallers) && sigprofCallers[n] != 0 {
@@ -3379,7 +3390,9 @@ func sigprofNonGo() {
//go:nosplit
//go:nowritebarrierrec
func sigprofNonGoPC(pc uintptr) {
+ SigprofNonGoPC = true
if prof.hz != 0 {
+ SigprofNonGoPCHz0 = true
pc := []uintptr{
pc,
funcPC(_ExternalCode) + sys.PCQuantum,
@@ -3390,6 +3403,7 @@ func sigprofNonGoPC(pc uintptr) {
osyield()
}
if prof.hz != 0 {
+ SigprofNonGoPCAddNonGo = true
cpuprof.addNonGo(pc)
}
atomic.Store(&prof.lock, 0)
diff --git a/src/runtime/testdata/testprogcgo/pprof.go b/src/runtime/testdata/testprogcgo/pprof.go
index 4460b93..64486ae 100644
--- a/src/runtime/testdata/testprogcgo/pprof.go
+++ b/src/runtime/testdata/testprogcgo/pprof.go
@@ -94,4 +94,11 @@ func CgoPprof() {
}
fmt.Println(name)
+ fmt.Fprintln(os.Stderr, "Sigprof ", runtime.Sigprof)
+ fmt.Fprintln(os.Stderr, "SigprofHz0 ", runtime.SigprofHz0)
+ fmt.Fprintln(os.Stderr, "SigprofPCMoreFMD ", runtime.SigprofPCMoreFMD)
+ fmt.Fprintln(os.Stderr, "SigprofNonGo ", runtime.SigprofNonGo)
+ fmt.Fprintln(os.Stderr, "SigprofNonGoPC ", runtime.SigprofNonGoPC)
+ fmt.Fprintln(os.Stderr, "SigprofNonGoPCHz0 ", runtime.SigprofNonGoPCHz0)
+ fmt.Fprintln(os.Stderr, "SigprofNonGoPCAddNonGo ", runtime.SigprofNonGoPCAddNonGo)
}I've observed following things. Given the following output: Hope this helps. |
|
Thanks. I do not understand what is different about your system that causes that code path to happen so frequently. |
|
@ianlancetaylor, I just switched paravirtualization interface to KVM on VirtualBox VM inside of which this tests failed and miraculously |
|
@codesenberg, @ianlancetaylor, is there anything more to do on this issue? It sounds like maybe it was a KVM problem? |
|
I never understood this issue. I'm fine with closing it. |
|
Yeah, looks like an issue with VirtualBox to me. I don't think there is anything to be done, so, closing it. |
What version of Go are you using (
go version)?go version devel +4cffe2b Sun Jan 29 23:31:20 2017 +0000 linux/amd64bootstrapped with:
go version go1.8rc3 linux/amd64What operating system and processor architecture are you using (
go env)?What did you do?
Tried to build Go from source and ran
all.bash.What did you expect to see?
Tests to pass.
What did you see instead?
So far I have seen three subtly different ways in which tests failed: first, second and third one.
Here are binaries and corresponding profiles.
I ran
all.bashseveral times and it failed consistently.The text was updated successfully, but these errors were encountered: