cmd/compile: FreeBSD, OpenBSD ARM builders failing with SIGBUS #15902

paulzhol · 2016-05-31T14:46:32Z

FreeBSD failures usually look like this:

# go run run.go -- fixedbugs/bug260.go
exit status 1
unexpected fault address 0x30354036
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0x30354036 pc=0x1107c]

goroutine 1 [running]:
runtime.throw(0xb887e, 0x5)
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-cedc7c8f2091/go/src/runtime/panic.go:566 +0x78 fp=0x30333e70 sp=0x30333e64
runtime.sigpanic()
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-cedc7c8f2091/go/src/runtime/sigpanic_unix.go:21 +0x1b4 fp=0x30333e9c sp=0x30333e70
main.main()
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-cedc7c8f2091/go/test/fixedbugs/bug260.go:30 +0x7c fp=0x30333f9c sp=0x30333ea0
runtime.main()
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-cedc7c8f2091/go/src/runtime/proc.go:183 +0x264 fp=0x30333fc4 sp=0x30333f9c
runtime.goexit()
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-cedc7c8f2091/go/src/runtime/asm_arm.s:998 +0x4 fp=0x30333fc4 sp=0x30333fc4
exit status 2

FAIL    fixedbugs/bug260.go 11.788s
2016/05/29 06:51:01 Failed: exit status 1

OpenBSD failures look like this (net/http tests):

ok      net 4.059s
unexpected fault address 0x10d60496
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0x10d60496 pc=0x12fdec]

goroutine 133 [running]:
runtime.throw(0x435815, 0x5)
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/runtime/panic.go:566 +0x78 fp=0x10aede2c sp=0x10aede20
runtime.sigpanic()
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/runtime/sigpanic_unix.go:21 +0x1b4 fp=0x10aede58 sp=0x10aede2c
net/http_test.testStreamingGet(0x10d39880, 0x32c66001)
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/net/http/client_test.go:599 +0x204 fp=0x10aedf8c sp=0x10aede5c
net/http_test.TestStreamingGet_h2(0x10d39880)
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/net/http/client_test.go:580 +0x24 fp=0x10aedf98 sp=0x10aedf8c
testing.tRunner(0x10d39880, 0x48cdc4)
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/testing/testing.go:610 +0xa4 fp=0x10aedfbc sp=0x10aedf98
runtime.goexit()
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/runtime/asm_arm.s:998 +0x4 fp=0x10aedfbc sp=0x10aedfbc
created by testing.(*T).Run
    /home/gopher/build/openbsd-arm-20803b845f26/go/src/testing/testing.go:646 +0x304
...

And also occasional happens on the FreeBSD builder:

ok      net 4.125s
unexpected fault address 0x30861d76
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0x30861d76 pc=0x1278f0]

goroutine 157 [running]:
runtime.throw(0x43096e, 0x5)
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/runtime/panic.go:566 +0x78 fp=0x309f5e2c sp=0x309f5e20
runtime.sigpanic()
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/runtime/sigpanic_unix.go:21 +0x1b4 fp=0x309f5e58 sp=0x309f5e2c
net/http_test.testStreamingGet(0x30829480, 0x282e1601)
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/net/http/client_test.go:599 +0x204 fp=0x309f5f8c sp=0x309f5e5c
net/http_test.TestStreamingGet_h2(0x30829480)
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/net/http/client_test.go:580 +0x24 fp=0x309f5f98 sp=0x309f5f8c
testing.tRunner(0x30829480, 0x486d70)
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/testing/testing.go:610 +0xa4 fp=0x309f5fbc sp=0x309f5f98
runtime.goexit()
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/runtime/asm_arm.s:998 +0x4 fp=0x309f5fbc sp=0x309f5fbc
created by testing.(*T).Run
    /usr/home/paulzhol/buildroot/freebsd-arm-paulzhol-8e6e9e838185/go/src/testing/testing.go:646 +0x304

I suspect http://golang.org/cl/23393 is the cause because it looks like the first occurrence on the OpenBSD builder:
https://build.golang.org/log/b75786ad6add5fe0116889706211d1e8faea9df5
With the FreeBSD builder being a bit more flaky, it starts happening after the e9228dd commit.

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2016-05-31T19:43:18Z

CC @randall77

randall77 · 2016-05-31T21:25:24Z

This is a problem at the intersection of tinyalloc, zeroing code, and alignment faults.

       0x005c 00092 (../test/fixedbugs/bug260.go:30)   MOVW    $type.[10]"".T1(SB), R0
        0x0060 00096 (../test/fixedbugs/bug260.go:30)   MOVW    R0, 4(R13)
        0x0064 00100 (../test/fixedbugs/bug260.go:30)   CALL    runtime.newobject(SB)
        0x0068 00104 (../test/fixedbugs/bug260.go:30)   MOVW    8(R13), R2
        0x006c 00108 (../test/fixedbugs/bug260.go:30)   MOVW    R2, R1
        ...
        0x0078 00120 (../test/fixedbugs/bug260.go:30)   MOVW    $0, R0
        0x007c 00124 (../test/fixedbugs/bug260.go:30)   MOVW.P  R0, 4(R1)
        0x0080 00128 (../test/fixedbugs/bug260.go:30)   MOVW.P  R0, 4(R1)
        0x0084 00132 (../test/fixedbugs/bug260.go:30)   MOVB.P  R0, 1(R1)
        0x0088 00136 (../test/fixedbugs/bug260.go:30)   MOVB.P  R0, 1(R1)

Those are 4-byte writes there, but mallocgc is returning something that isn't 4-byte aligned. It is doing that because [10]byte is tiny-allocatable, so sometimes we get an allocation some way into a 16-byte tinyalloc chunk.

The fix should be to do zeroing correctly, respecting the possible alignment of the type.

Nagging questions TBA:
Why doesn't this error happen on linux/arm?
Why does this have anything to do with the live variable analysis change?

randall77 · 2016-05-31T21:48:14Z

Why does this have anything to do with the live variable analysis change?

Before the liveness change, we relied on newobject to return zeroed memory, so the compiler did not generate code to zero it again. After the liveness change, the compiler emits a useless zeroing. Bad, but not fix-for-go1.7 bad. The zeroing-generating code itself is to blame.

mdempsky · 2016-05-31T21:49:43Z

Why doesn't this error happen on linux/arm?

My understanding is Linux on ARM supports emulating unaligned memory accesses (to make porting programs from x86 easier), and it's possible the builders have that option enabled.

I don't believe OpenBSD implements that emulation, and it seems likely that FreeBSD doesn't either.

randall77 · 2016-05-31T21:58:35Z

Fix mailed (CL 23587)

I added a test. If anyone has access to a freebsd/arm box, please try the test without this CL and see if it breaks. It should. I'm kind of baffled as to how freebsd/arm ever used to work before the fix - I guess we just don't ever zero unaligned memory.

gopherbot · 2016-05-31T22:00:20Z

CL https://golang.org/cl/23587 mentions this issue.

mdempsky · 2016-05-31T22:05:10Z

@randall77 I think only tinyalloc ever returns unaligned memory, and it's always zero'd.

randall77 · 2016-05-31T22:05:58Z

Right, but see my test, it isn't just malloc where we zero memory.

mdempsky · 2016-05-31T22:08:50Z

@randall77 Hm, yeah, that is baffling.

davecheney · 2016-05-31T23:20:53Z

Keith, Linux/arm doesn't fault on unaligned memory access, you just pay the
penalty for the double load silently. The only platform we had that
enforced this was the original arm5 builder that died last year.

On Wed, 1 Jun 2016, 08:08 Matthew Dempsky notifications@github.com wrote:

@randall77 https://github.com/randall77 Hm, yeah, that is baffling.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#15902 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAAcA8mA6Rutaq8FiRit9o72zaFoJ-wvks5qHLF0gaJpZM4IqnEU
.

paulzhol · 2016-06-01T07:14:17Z

I ran the test with go version go1.6.2 freebsd/arm, it indeed fails:

./issue15902
unexpected fault address 0xaebd9
fatal error: fault
[signal 0xa code=0x1 addr=0xaebd9 pc=0x11018]

goroutine 1 [running]:
runtime.throw(0x70998, 0x5)
        /usr/home/paulzhol/go1.6/src/runtime/panic.go:547 +0x78 fp=0x301cff64 sp=0x301cff58
runtime.sigpanic()
        /usr/home/paulzhol/go1.6/src/runtime/sigpanic_unix.go:21 +0x1b4 fp=0x301cff90 sp=0x301cff64
main.f(0xaebd8)
        /usr/home/paulzhol/issue15902.go:20 +0x18 fp=0x301cff94 sp=0x301cff94
main.main()
        /usr/home/paulzhol/issue15902.go:26 +0x1c fp=0x301cff9c sp=0x301cff94
runtime.main()
        /usr/home/paulzhol/go1.6/src/runtime/proc.go:188 +0x2b4 fp=0x301cffc4 sp=0x301cff9c
runtime.goexit()
        /usr/home/paulzhol/go1.6/src/runtime/asm_arm.s:990 +0x4 fp=0x301cffc4 sp=0x301cffc4

same with tip go version devel +88ae649 Wed Jun 1 04:27:28 2016 +0000 freebsd/arm:

./issue15902.tip 
unexpected fault address 0xa8e01
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0xa8e01 pc=0x11018]

goroutine 1 [running]:
runtime.throw(0x75fac, 0x5)
        /tmp/go/src/runtime/panic.go:566 +0x78 fp=0x302cbf64 sp=0x302cbf58
runtime.sigpanic()
        /tmp/go/src/runtime/sigpanic_unix.go:21 +0x1b4 fp=0x302cbf90 sp=0x302cbf64
main.f(0xa8e00)
        /usr/home/paulzhol/issue15902.go:20 +0x18 fp=0x302cbf94 sp=0x302cbf94
main.main()
        /usr/home/paulzhol/issue15902.go:26 +0x1c fp=0x302cbf9c sp=0x302cbf94
runtime.main()
        /tmp/go/src/runtime/proc.go:183 +0x264 fp=0x302cbfc4 sp=0x302cbf9c
runtime.goexit()
        /tmp/go/src/runtime/asm_arm.s:998 +0x4 fp=0x302cbfc4 sp=0x302cbfc4

And just before http://golang.org/cl/23393 go version devel +99d29d5 Fri May 27 01:55:31 2016 +0000 freebsd/arm:

./issue15902.before_liveness
unexpected fault address 0xa8e01
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0xa8e01 pc=0x11018]

goroutine 1 [running]:
runtime.throw(0x75f94, 0x5)
        /tmp/go/src/runtime/panic.go:566 +0x78 fp=0x302cbf64 sp=0x302cbf58
runtime.sigpanic()
        /tmp/go/src/runtime/sigpanic_unix.go:21 +0x1b4 fp=0x302cbf90 sp=0x302cbf64
main.f(0xa8e00)
        /usr/home/paulzhol/issue15902.go:20 +0x18 fp=0x302cbf94 sp=0x302cbf94
main.main()
        /usr/home/paulzhol/issue15902.go:26 +0x1c fp=0x302cbf9c sp=0x302cbf94
runtime.main()
        /tmp/go/src/runtime/proc.go:183 +0x264 fp=0x302cbfc4 sp=0x302cbf9c
runtime.goexit()
        /tmp/go/src/runtime/asm_arm.s:998 +0x4 fp=0x302cbfc4 sp=0x302cbfc4

Applying CL https://golang.org/cl/23587 fixes it.

ianlancetaylor changed the title ~~FreeBSD, OpenBSD ARM builders failing with SIGBUS~~ cmd/compile: FreeBSD, OpenBSD ARM builders failing with SIGBUS May 31, 2016

ianlancetaylor added this to the Go1.7 milestone May 31, 2016

randall77 modified the milestones: Go1.7Beta, Go1.7 May 31, 2016

randall77 mentioned this issue May 31, 2016

cmd/compile: unnecessary zeroing #15914

Closed

gopherbot closed this as completed in 52fe472 Jun 1, 2016

golang locked and limited conversation to collaborators Jun 1, 2017

gopherbot added the FrozenDueToAge label Jun 1, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/compile: FreeBSD, OpenBSD ARM builders failing with SIGBUS #15902

cmd/compile: FreeBSD, OpenBSD ARM builders failing with SIGBUS #15902

paulzhol commented May 31, 2016

ianlancetaylor commented May 31, 2016

randall77 commented May 31, 2016

randall77 commented May 31, 2016

mdempsky commented May 31, 2016

randall77 commented May 31, 2016

gopherbot commented May 31, 2016

mdempsky commented May 31, 2016

randall77 commented May 31, 2016

mdempsky commented May 31, 2016

davecheney commented May 31, 2016

paulzhol commented Jun 1, 2016 •

edited

cmd/compile: FreeBSD, OpenBSD ARM builders failing with SIGBUS #15902

cmd/compile: FreeBSD, OpenBSD ARM builders failing with SIGBUS #15902

Comments

paulzhol commented May 31, 2016

ianlancetaylor commented May 31, 2016

randall77 commented May 31, 2016

randall77 commented May 31, 2016

mdempsky commented May 31, 2016

randall77 commented May 31, 2016

gopherbot commented May 31, 2016

mdempsky commented May 31, 2016

randall77 commented May 31, 2016

mdempsky commented May 31, 2016

davecheney commented May 31, 2016

paulzhol commented Jun 1, 2016 • edited

paulzhol commented Jun 1, 2016 •

edited