Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: issues on linux/386 #5007

Closed
alberts opened this issue Mar 8, 2013 · 11 comments
Closed

runtime: issues on linux/386 #5007

alberts opened this issue Mar 8, 2013 · 11 comments
Milestone

Comments

@alberts
Copy link
Contributor

alberts commented Mar 8, 2013

What steps will reproduce the problem?

the time tests are pretty good at blowing up linux/386

#!/bin/bash
set -xe
export GOARCH=386
export GOHOSTARCH=386
go version
go test -c time
while true; do
GOMAXPROCS=45 ./time.test
done

What do you see instead?

split stack overflows, nil pointer dereferences

example at the bottom

might also be the cause of issue #5005

Which compiler are you using (5g, 6g, 8g, gccgo)?

8g

Which operating system are you using?

linux

Which version are you using?  (run 'go version')

go version devel +ed48ace1855b Thu Mar 07 20:11:37 2013 -0800 linux/386

Please provide any additional information below.

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x5 pc=0x80662a5]

goroutine 3 [running]:

goroutine 1 [chan receive]:
testing.RunTests(0x819da4c, 0x8249320, 0x35, 0x35, 0x1, ...)
    /build/go.386/go/src/pkg/testing/testing.go:427 +0x69f
testing.Main(0x819da4c, 0x8249320, 0x35, 0x35, 0x8247960, ...)
    /build/go.386/go/src/pkg/testing/testing.go:358 +0x69
main.main()
    time/_test/_testmain.go:185 +0x81

goroutine 18 [chan receive]:
time_test.TestAfterStress(0x183b4000)
    /build/go.386/go/src/pkg/time/sleep_test.go:65 +0x9d
testing.tRunner(0x183b4000, 0x8249338)
    /build/go.386/go/src/pkg/testing/testing.go:346 +0x87
created by testing.RunTests
    /build/go.386/go/src/pkg/testing/testing.go:426 +0x684

goroutine 7 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 8 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 9 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 10 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 11 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 12 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 13 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 14 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 15 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 16 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·004()
    /build/go.386/go/src/pkg/time/sleep_test.go:42 +0x63
created by time.goFunc
    /build/go.386/go/src/pkg/time/sleep.go:122 +0x45

goroutine 19 [sleep]:
time.Sleep(0x1, 0x0)
    /build/go.386/go/src/pkg/runtime/ztime_linux_386.c:19 +0x3a
time_test.func·005()
    /build/go.386/go/src/pkg/time/sleep_test.go:60 +0x4f
created by time_test.TestAfterStress
    /build/go.386/go/src/pkg/time/sleep_test.go:62 +0x60
FAIL    time    0.128s
@ianlancetaylor
Copy link
Contributor

Comment 1:

Labels changed: added priority-asap, go1.1, removed priority-triage.

@gopherbot
Copy link

Comment 2:

Does the issue occur if you put "GOGC=off GOMAXPROCS=45 ./time.test" in the script?

@alberts
Copy link
Contributor Author

alberts commented Mar 8, 2013

Comment 3:

testing now

@dvyukov
Copy link
Member

dvyukov commented Mar 8, 2013

Comment 4:

To make it clear, it happens only on 386, right? amd64 is not affected?

@alberts
Copy link
Contributor Author

alberts commented Mar 8, 2013

Comment 5:

I've never seen this on amd64. I've done about 6000 builds on amd64.
I found this within 20 builds on 386.
So I'd say that amd64 isn't affected, but I'll specifically run the time tests for a
while.

@alberts
Copy link
Contributor Author

alberts commented Mar 8, 2013

Comment 6:

I've been running the time tests on 386 for about 2 hours with GOGC=off and haven't seen
a crash yet. I'll keep it running. Seems likely that it's GC related though.

@alberts
Copy link
Contributor Author

alberts commented Mar 8, 2013

Comment 7:

I've also seen timeouts of runtime tests TestGoexitDeadlock and TestInitDeadlock.

@alberts
Copy link
Contributor Author

alberts commented Mar 8, 2013

Comment 8:

I've also seen timeouts of runtime tests TestGoexitDeadlock and TestInitDeadlock.

@alberts
Copy link
Contributor Author

alberts commented Mar 9, 2013

Comment 9:

Okay, so far I've seen the following:
linux/386: issue #5005, net/http tests failing, never seen this on amd64
linux/386: issue #5007, time tests failing if GC is enabled, works if GC is disabled,
tried to reproduce on amd64 but it seems fine there
linux/386 and linux/amd64: go test -v -cpu 1,2,4 -timeout=30m runtime tests timing out.
so far I've seen TestGoexitDeadlock and TestInitDeadlock on 386 and TestLockedDeadlock2
on amd64 time out
linux/386: I can reproduce the problem in issue #4869
linux/386 and linux/amd64: issue #4987: os/signal.TestSignal failures on both
architectures. I've only been able to reproduce this on one machine though, so I don't
know if it's the machine or the kernel maybe. strace looks sane though.
Here's a stacktrace for the runtime test deadlock on amd64:
go version devel +80a76d7f37a1 Fri Mar 08 16:39:54 2013 -0800 linux/amd64
=== RUN TestLockedDeadlock2
panic: test timed out
goroutine 77 [running]:
testing.alarm()
    /build/go.tip/go/src/pkg/testing/testing.go:526 +0x53
created by time.goFunc
    /build/go.tip/go/src/pkg/time/sleep.go:122 +0x48
goroutine 1 [chan receive]:
testing.RunTests(0x6359e0, 0x74bac0, 0x22, 0x22, 0x1, ...)
    /build/go.tip/go/src/pkg/testing/testing.go:427 +0x88e
testing.Main(0x6359e0, 0x74bac0, 0x22, 0x22, 0x74d540, ...)
    /build/go.tip/go/src/pkg/testing/testing.go:358 +0x8a
main.main()
    runtime/_test/_testmain.go:251 +0x9a
goroutine 58 [syscall]:
syscall.Syscall6()
    /build/go.tip/go/src/pkg/syscall/asm_linux_amd64.s:41 +0x5
syscall.wait4(0x431f, 0xc200000b40, 0x0, 0xc200123e10, 0x100000001, ...)
    /build/go.tip/go/src/pkg/syscall/zerrors_linux_amd64.go:1640 +0x7b
syscall.Wait4(0x431f, 0x7f650072ab44, 0x0, 0xc200123e10, 0xc20012be70, ...)
    /build/go.tip/go/src/pkg/syscall/syscall_linux.go:222 +0x6c
os.(*Process).wait(0xc20012fee0, 0x0, 0x0, 0x0)
    /build/go.tip/go/src/pkg/os/exec_unix.go:22 +0xea
os.(*Process).Wait(0xc20012fee0, 0xc200000ab8, 0x0, 0x0)
    /build/go.tip/go/src/pkg/os/doc.go:43 +0x25
os/exec.(*Cmd).Wait(0xc200134140, 0x0, 0x0)
    /build/go.tip/go/src/pkg/os/exec/exec.go:308 +0x197
os/exec.(*Cmd).Run(0xc200134140, 0xc20012a540, 0xc20012be70)
    /build/go.tip/go/src/pkg/os/exec/exec.go:232 +0x52
os/exec.(*Cmd).CombinedOutput(0xc200134140, 0x2, 0x7f650072adc0, 0x2, 0x2, ...)
    /build/go.tip/go/src/pkg/os/exec/exec.go:352 +0x20d
runtime_test.executeTest(0xc200123b40, 0x63b130, 0x9c, 0x0, 0x0, ...)
    /build/go.tip/go/src/pkg/runtime/crash_test.go:40 +0x61f
runtime_test.testDeadlock(0xc200123b40, 0x63b130, 0x9c)
    /build/go.tip/go/src/pkg/runtime/crash_test.go:71 +0x56
runtime_test.TestLockedDeadlock2(0xc200123b40)
    /build/go.tip/go/src/pkg/runtime/crash_test.go:91 +0x39
testing.tRunner(0xc200123b40, 0x74bbc8)
    /build/go.tip/go/src/pkg/testing/testing.go:346 +0x8a
created by testing.RunTests
    /build/go.tip/go/src/pkg/testing/testing.go:426 +0x86b
goroutine 63 [finalizer wait]:
goroutine 61 [syscall]:
syscall.Syscall()
    /build/go.tip/go/src/pkg/syscall/asm_linux_amd64.s:16 +0x5
syscall.read(0x4, 0xc200130600, 0x200, 0x200, 0x8, ...)
    /build/go.tip/go/src/pkg/syscall/zerrors_linux_amd64.go:2337 +0x70
syscall.Read(0x4, 0xc200130600, 0x200, 0x200, 0x413b44, ...)
    /build/go.tip/go/src/pkg/syscall/syscall_unix.go:132 +0x5a
os.(*File).read(0xc200000ae8, 0xc200130600, 0x200, 0x200, 0x55ab40, ...)
    /build/go.tip/go/src/pkg/os/file_unix.go:174 +0x60
os.(*File).Read(0xc200000ae8, 0xc200130600, 0x200, 0x200, 0x0, ...)
    /build/go.tip/go/src/pkg/os/file.go:95 +0x96
bytes.(*Buffer).ReadFrom(0xc20012a540, 0xc2000c9d80, 0xc200000ae8, 0x0, 0x0, ...)
    /build/go.tip/go/src/pkg/bytes/buffer.go:166 +0x203
io.Copy(0xc2000c9ab0, 0xc20012a540, 0xc2000c9d80, 0xc200000ae8, 0x0, ...)
    /build/go.tip/go/src/pkg/io/io.go:328 +0xa3
os/exec.func·003(0x63b1cc, 0xc20012a3f0)
    /build/go.tip/go/src/pkg/os/exec/exec.go:207 +0x60
os/exec.func·004(0xc20012fc20)
    /build/go.tip/go/src/pkg/os/exec/exec.go:274 +0x2a
created by os/exec.(*Cmd).Start
    /build/go.tip/go/src/pkg/os/exec/exec.go:275 +0x6ac
FAIL    runtime 1800.050s

@dvyukov
Copy link
Member

dvyukov commented Mar 10, 2013

Comment 10:

mailed https://golang.org/cl/7534044/

Owner changed to @dvyukov.

Status changed to Accepted.

@dvyukov
Copy link
Member

dvyukov commented Mar 10, 2013

Comment 11:

This issue was closed by revision 433824d.

Status changed to Fixed.

@rsc rsc added this to the Go1.1 milestone Apr 14, 2015
@rsc rsc removed the go1.1 label Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants