Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime/pprof: TestStackBarrierProfiling fails with "SIGTRAP: trace trap PC=0x459cfd m=2" on linux #13864

Closed
mikioh opened this issue Jan 8, 2016 · 4 comments
Closed

runtime/pprof: TestStackBarrierProfiling fails with "SIGTRAP: trace trap PC=0x459cfd m=2" on linux #13864

mikioh opened this issue Jan 8, 2016 · 4 comments
Labels
FrozenDueToAge
Milestone
Go1.6Maybe

Comments

@mikioh
Copy link
Contributor

mikioh commented Jan 8, 2016

all.bash fails on fedora-vm,

uname -a
Linux fedora-amd64.localdomain 4.2.8-200.fc22.x86_64 #1 SMP Tue Dec 15 16:50:23 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

git rev-parse HEAD
acc7161320038ee6377d589890526cd11912b4e8

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7851
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

with the following dying message:

--- FAIL: TestStackBarrierProfiling (0.02s)
    pprof_test.go:363: subprocess failed with exit status 2:
        SIGTRAP: trace trap
        PC=0x459cfd m=2

        goroutine 0 [idle]:
        runtime.morestack()
            /go/src/runtime/asm_amd64.s:331 +0x1d

        goroutine 4 [running]:
        runtime.systemstack_switch()
            /go/src/runtime/asm_amd64.s:245 fp=0xc8200414f8 sp=0xc8200414f0
        runtime.gcMarkTermination()
            /go/src/runtime/mgc.go:1255 +0x3a9 fp=0xc820041748 sp=0xc8200414f8
        runtime.gcMarkDone()
            /go/src/runtime/mgc.go:1141 +0x286 fp=0xc820041768 sp=0xc820041748
        runtime.gcBgMarkWorker(0xc82001c000)
            /go/src/runtime/mgc.go:1465 +0x4d8 fp=0xc8200417d8 sp=0xc820041768
        runtime.goexit()
            /go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc8200417e0 sp=0xc8200417d8
        created by runtime.gcBgMarkStartWorkers
            /go/src/runtime/mgc.go:1335 +0x93

        goroutine 1 [chan receive]:
        testing.RunTests(0x610ac0, 0x6b03e0, 0x8, 0x8, 0x7f2e00000001)
            /go/src/testing/testing.go:583 +0x8d2
        testing.(*M).Run(0xc8200bef38, 0x408ee7)
            /go/src/testing/testing.go:515 +0x81
        main.main()
            runtime/pprof/_test/_testmain.go:70 +0x117

        goroutine 19 [runnable]:
        runtime/pprof_test.deepStack(0x1c1, 0xc820161800)
            /go/src/runtime/pprof/pprof_test.go:393 +0x52
        runtime/pprof_test.deepStack(0x1c2, 0xc820161400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c3, 0xc820161000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c4, 0xc820160c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c5, 0xc820160800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c6, 0xc820160400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c7, 0xc820160000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c8, 0xc82015fc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1c9, 0xc82015f800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ca, 0xc82015f400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1cb, 0xc82015f000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1cc, 0xc82015ec00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1cd, 0xc82015e800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ce, 0xc82015e400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1cf, 0xc82015e000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d0, 0xc82015dc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d1, 0xc82015d800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d2, 0xc82015d400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d3, 0xc82015d000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d4, 0xc82015cc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d5, 0xc82015c800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d6, 0xc82015c400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d7, 0xc82015c000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d8, 0xc82015bc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1d9, 0xc82015b800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1da, 0xc82015b400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1db, 0xc82015b000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1dc, 0xc82015ac00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1dd, 0xc82015a800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1de, 0xc82015a400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1df, 0xc82015a000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e0, 0xc820159c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e1, 0xc820159800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e2, 0xc820159400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e3, 0xc820159000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e4, 0xc820158c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e5, 0xc820158800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e6, 0xc820158400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e7, 0xc820158000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e8, 0xc820157c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1e9, 0xc820157800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ea, 0xc820157400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1eb, 0xc820157000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ec, 0xc820156c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ed, 0xc820156800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ee, 0xc820156400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ef, 0xc820156000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f0, 0xc820155c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f1, 0xc820155800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f2, 0xc820155400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f3, 0xc820155000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f4, 0xc820154c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f5, 0xc820154800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f6, 0xc820154400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f7, 0xc820154000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f8, 0xc820153c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1f9, 0xc820153800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1fa, 0xc820153400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1fb, 0xc820153000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1fc, 0xc820152c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1fd, 0xc820152800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1fe, 0xc820152400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x1ff, 0xc820152000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x200, 0xc820151c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x201, 0xc820151800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x202, 0xc820151400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x203, 0xc820151000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x204, 0xc820150c00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x205, 0xc820150800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x206, 0xc820150400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x207, 0xc820150000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x208, 0xc82014fc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x209, 0xc82014f800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x20a, 0xc82014f400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x20b, 0xc82014f000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x20c, 0xc82014ec00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x20d, 0xc82014e800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x20e, 0xc82014e400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x20f, 0xc82014e000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x210, 0xc82014dc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x211, 0xc82014d800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x212, 0xc82014d400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x213, 0xc82014d000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x214, 0xc82014cc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x215, 0xc82014c800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x216, 0xc82014c400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x217, 0xc82014c000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x218, 0xc82014bc00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x219, 0xc82014b800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x21a, 0xc82014b400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x21b, 0xc82014b000)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x21c, 0xc82014ac00)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x21d, 0xc82014a800)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        runtime/pprof_test.deepStack(0x21e, 0xc82014a400)
            /go/src/runtime/pprof/pprof_test.go:394 +0x90
        created by testing.RunTests
            /go/src/testing/testing.go:582 +0x892

        goroutine 20 [runnable]:
        runtime/pprof.profileWriter(0x7f2e60199650, 0xc82009a310)
            /go/src/runtime/pprof/pprof.go:597
        created by runtime/pprof.StartCPUProfile
            /go/src/runtime/pprof/pprof.go:593 +0x145

        rax    0xf80
        rbx    0xc82002a400
        rcx    0xc820000900
        rdx    0x0
        rdi    0x42eec0
        rsi    0xc820000900
        rbp    0x2
        rsp    0xc82003af78
        r8     0xc820000f00
        r9     0x47ae147ae147ae15
        r10    0x4d9e5
        r11    0x203
        r12    0x42f150
        r13    0x60fe08
        r14    0x0
        r15    0x8
        rip    0x459cfd
        rflags 0x246
        cs     0x33
        fs     0x0
        gs     0x0
FAIL
@mikioh mikioh added this to the Go1.6 milestone Jan 8, 2016
@rsc
Copy link
Contributor

rsc commented Jan 14, 2016

Pretty sure this is a duplicate of #13363. Not that that helps. /cc @aclements

@rsc rsc modified the milestones: Go1.6Maybe, Go1.6 Jan 14, 2016
@aclements
Copy link
Member

I actually don't think this is a duplicate of #13363. It's crashing a few lines after morestack successfully got the m, and is indicating we blew the system stack. Unfortunately, the traceback on the system stack cut off at the morestack right at the top because that's normally the right thing to do, but it hid whatever was overflowing the system stack.

@mikioh, are you able to reproduce this? If so, could you try https://go-review.googlesource.com/18761, which should fix #13363 (but I suspect won't fix this).

@mikioh
Copy link
Contributor Author

mikioh commented Jan 20, 2016

are you able to reproduce this?

Yesterday I tried and failed to reproduce it. Also I've stashed my Linux VM for investigating signal stack setup issue on NetBSD and DragonFly BSD.

@aclements
Copy link
Member

Now that I understand the cause of #13363 (the same G running on two Ms at once), I think it is possible this had the same root cause, mostly because pretty much anything could happen once it got in to that state and the traceback shows it's clearly around the area where that problem occurred.

I'm going to close this as a dup of #13363, but if it happens again, please reopen.

@AkihiroSuda AkihiroSuda mentioned this issue Apr 11, 2016
Closed
@golang golang locked and limited conversation to collaborators Feb 3, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1.6Maybe
Development

No branches or pull requests
4 participants
@mikioh @rsc @aclements @gopherbot