Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: go 1.6 segfault with musl libc #14476

Closed
nathanejohnson opened this issue Feb 23, 2016 · 19 comments
Closed

runtime: go 1.6 segfault with musl libc #14476

nathanejohnson opened this issue Feb 23, 2016 · 19 comments

Comments

@nathanejohnson
Copy link

I am attempting to compile go 1.6 from a go 1.4.3 bootstrap. The build environment is Alpine linux 3.3.1 with the vanilla (i.e., no grsec) 4.1.15 kernel. The libc is musl. Go gets compiled using the build.bash script, and no errors are thrown. However, the resulting binary gives a segmentation fault whenever I attempt to use it.

abuildx86:~/aports/community/go/src/go/bin$ strace ./go
execve("./go", ["./go"], [/* 18 vars */]) = 0
set_thread_area({entry_number:-1, base_addr:0xb7768844, limit:1048575, seg_32bit:1, contents:0,      read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 (entry_number:6)
set_tid_address(0xb7768860)             = 29565
mprotect(0xb7767000, 4096, PROT_READ)   = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0} ---
+++ killed by SIGSEGV +++
Segmentation fault

abuildx86:~/aports/community/go/src/go/bin$ uname -a
Linux abuildx86 4.1.15 #1-Alpine SMP Tue Dec 15 09:29:30 GMT 2015 i686 Linux
abuildx86:~/aports/community/go/src/go/bin$ ldd go
    /lib/ld-musl-i386.so.1 (0x80001000)
    libc.musl-x86.so.1 => /lib/ld-musl-i386.so.1 (0x80001000)
abuildx86:~/aports/community/go/src/go/bin$ 

This same environment has no problems compiling go 1.5.3 from source. My other alpine 3.3.1 VM with amd64 also has no issues compiling / running 1.5.3 or 1.6, even with the grsec kernel.

Any thoughts?

@ianlancetaylor ianlancetaylor changed the title go 1.6 segfault runtime: go 1.6 segfault Feb 23, 2016
@ianlancetaylor ianlancetaylor added this to the Go1.6.1 milestone Feb 23, 2016
@ianlancetaylor
Copy link
Contributor

Looks like a nil pointer dereference, but we don't know where it is happening. Can you run the executable under gdb, and see what it says about where the signal occurs? Try to get a backtrace.

@minux
Copy link
Member

minux commented Feb 23, 2016

interesting.

could you please run go under gdb and see where does it crash?

I wonder if this is related to musl libc. Does gofmt (which should be a
static binary) crash?

The ldd output looks wrong too. At least bin/go should link to libc.so
and libpthread.so.

@bradfitz, we might need to run a musl-based builder.

@nathanejohnson
Copy link
Author

abuildx86:~/aports/community/go/src/go/bin$ gdb ./go
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i586-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./go...done.
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /home/njohnson/aports/community/go/src/go/bin/go.
Use `info auto-load python-scripts [REGEXP]' to list them.
(gdb) run
Starting program: /home/njohnson/aports/community/go/src/go/bin/go 

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) 

@nathanejohnson
Copy link
Author

gofmt is fine, but statically linked.

abuildx86:~/aports/community/go/src/go/bin$ file gofmt
gofmt: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
abuildx86:~/aports/community/go/src/go/bin$ 

@dominikh
Copy link
Member

The ldd output looks wrong too. At least bin/go should link to libc.so
and libpthread.so.

I can't spot anything wrong. With musl, libpthread, like other libraries, are all bundled into a single library. That includes the dynamic linker: http://wiki.musl-libc.org/wiki/Design_Concepts#Unified_libc.2Flibpthread.2Fldso

However, I've just tested Go 1.6 on the musl version of Void Linux and cannot reproduce the segfault. It is a 64 bit machine, unlike OP's 32 bit.

@nathanejohnson
Copy link
Author

dominikh, thanks for confirming. I cannot reproduce this on amd64 either, same kernel version on same distribution with same version musl.

@minux
Copy link
Member

minux commented Feb 23, 2016 via email

@nathanejohnson
Copy link
Author

minux, alpine 386 image would be appropriate I should think: http://www.alpinelinux.org/downloads/

get the non 64 bit one. of course,that will have grsec, and will make debugging harder . but once it installs you can install the vanilla kernel

@minux
Copy link
Member

minux commented Feb 23, 2016 via email

@minux
Copy link
Member

minux commented Feb 23, 2016

I tried two programs.

bin/go fails in runtime.sched_getaffinity (the very first syscall):
(gdb) r
Starting program: /root/go.git/bin/go

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) bt
#0 0x00000000 in ?? ()
#1 0x0810cfa8 in runtime.sched_getaffinity () at
/root/go.git/src/runtime/sys_linux_386.s:483
#2 0x080e15df in runtime.getproccount (~r0=-1073742492)
at /root/go.git/src/runtime/os1_linux.go:88
#3 0x080e180a in runtime.osinit () at
/root/go.git/src/runtime/os1_linux.go:172
#4 0x08109e13 in runtime.rt0_go () at
/root/go.git/src/runtime/asm_386.s:147
#5 0x00000001 in ?? ()
#6 0xbffffd64 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

It's because musl doesn't put vsyscall entry point in %gs:0x10.

An externally linked hello world program couldn't even start:
(gdb) r
Starting program: /root/go.git/src/hello

Program received signal SIGSEGV, Segmentation fault.
0xb7f98656 in do_relocs (dso=dso@entry=0xb7ffd660 , rel=0x80000910,
rel_size=145784, stride=2)
at src/ldso/dynlink.c:407
407 src/ldso/dynlink.c: No such file or directory.
(gdb) bt
#0 0xb7f98656 in do_relocs (dso=dso@entry=0xb7ffd660 ,
rel=0x80000910, rel_size=145784,
stride=2) at src/ldso/dynlink.c:407
#1 0xb7f98ea7 in reloc_all (p=p@entry=0xb7ffd660 ) at
src/ldso/dynlink.c:1115
#2 0xb7f9a5e3 in __dls3 (sp=0xbffffd60) at src/ldso/dynlink.c:1598
#3 0xb7f99e5c in __dls2 (base=0xb7f7b000 "\177ELF\001\001\001",
sp=0xbffffd60)
at src/ldso/dynlink.c:1389
#4 0xb7f97ec5 in _dlstart_c (sp=0xbffffd60, dynv=0xb7ffcef8) at
src/ldso/dlstart.c:147
#5 0xb7f97dbb in _dlstart () from /lib/ld-musl-i386.so.1

Something is wrong with the relocations.

@minux
Copy link
Member

minux commented Feb 23, 2016 via email

@minux minux changed the title runtime: go 1.6 segfault runtime: go 1.6 segfault with musl libc Feb 23, 2016
@gopherbot
Copy link

CL https://golang.org/cl/19833 mentions this issue.

ncopa pushed a commit to alpinelinux/aports that referenced this issue Feb 23, 2016
Including patch to fix 386 build.
golang/go#14476

Removed default-sc-getpw-r-size-max.patch, fixed upstream.
golang/go#11319

Changed the offset for no-pic.patch to match new code.

TT: pkgrel set to 1 as the original pkgrel=0 failed to build
with x86 and was reverted.
@cyli
Copy link

cyli commented Mar 17, 2016

Sorry, I think I'm late to the game, but I may also seeing this on linux/amd64? I'm trying to build https://github.com/mattes/migrate (which depends upon https://github.com/mattn/go-sqlite3, which has some CGO, but I don't know what to look for), and everything builds fine, but I'm getting segfaults as well by doing:

docker run -it golang:1.6.0-alpine sh
apk add --update git alpine-sdk
go get github.com/mattes/migrate
migrate

The 1.6.0-alpine image is based on alpine3.3, so maybe this can be replicated using http://wiki.alpinelinux.org/cgi-bin/dl.cgi/v3.3/releases/x86_64/alpine-3.3.1-x86_64.iso?

Here are the strace and GDB results:

$ strace /go/bin/migrate 
execve("/go/bin/migrate", ["/go/bin/migrate"], [/* 14 vars */]) = 0
arch_prctl(ARCH_SET_FS, 0x7fac21441da8) = 0
set_tid_address(0x7fac21441de0)         = 12
mprotect(0x7fac21440000, 4096, PROT_READ) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x55961ac54f00} ---
+++ killed by SIGSEGV +++
Segmentation fault

Output from running gdb /go/bin/migrate:

Reading symbols from /go/bin/migrate...done.
Loading Go Runtime support.
(gdb) r
Starting program: /go/bin/migrate 

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7d93531 in ?? () from /lib/ld-musl-x86_64.so.1
(gdb) bt
#0  0x00007ffff7d93531 in ?? () from /lib/ld-musl-x86_64.so.1
#1  0x0000000000000000 in ?? ()
(gdb) quit

(See mattes/migrate#76 (comment))

@ianlancetaylor
Copy link
Contributor

@cyli The fix for this issue is on tip but is not in 1.6.

If you have a different problem, please open a new issue. This one is closed. Thanks.

@cyli
Copy link

cyli commented Mar 18, 2016

@ianlancetaylor The fix on tip seems to be for 386 only and I am seeing this in amd64. I am happy to open a new issue though!

@minux
Copy link
Member

minux commented Mar 18, 2016 via email

@adg adg modified the milestones: Go1.6.2, Go1.6.1 Apr 7, 2016
@gopherbot
Copy link

CL https://golang.org/cl/22038 mentions this issue.

gopherbot pushed a commit that referenced this issue Apr 14, 2016
Like bionic, musl also doesn't provide vsyscall helper in %gs:0x10,
and as int $0x80 is as fast as calling %gs:0x10, just use int $0x80
always.

Because we're no longer using vsyscall in VDSO, get rid of VDSO code
for linux/386 too.

Fixes #14476.

Change-Id: I00ec8652060700e0a3c9b524bfe3c16a810263f6
Reviewed-on: https://go-review.googlesource.com/19833
Run-TryBot: Minux Ma <minux@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22038
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@sheldonkwok
Copy link

Is this supposed to be resolved in 1.6.2? I'm still facing similar segfault issues when using go-sqlite3.

@ianlancetaylor
Copy link
Contributor

@sheldonkwok This issue is believed to be fixed in 1.6.2. If you are having trouble, please open a new issue. A segfault can have many possible causes.

@golang golang locked and limited conversation to collaborators Jul 15, 2017
@rsc rsc unassigned minux Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

9 participants