Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

misc/cgo/testshared: segfaults on linux/arm64 Fedora 29 #31823

Closed
jcajka opened this issue May 3, 2019 · 14 comments
Closed

misc/cgo/testshared: segfaults on linux/arm64 Fedora 29 #31823

jcajka opened this issue May 3, 2019 · 14 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone
Go1.13

Comments

@jcajka
Copy link
Contributor

jcajka commented May 3, 2019

What version of Go are you using (go version)?

go version devel +004fb5cb8d Fri May 3 03:49:11 2019 +0000 linux/arm64

Does this issue reproduce with the latest release?

Does reproduce on master, Go 1.12 and Go 1.11 on Fedora 29.

What operating system and processor architecture are you using (go env)?

linux/arm64

What did you do?

I have built master branch with go1.11.6 on linux/arm64 host(Fedora 29) and

go tool dist test -v -v -v -k

This is with

gcc-8.3.1-2.fc29.aarch64
binutils-2.31.1-17.fc29.aarch64
glibc-2.28-27.fc29.aarch64

What did you expect to see?

Tests passing

What did you see instead?

##### ../misc/cgo/testshared
# go tool dist test -run=^testshared$
go test -short -count=1 -tags= -run= -timeout=10m0s
--- FAIL: TestTrivialExecutable (8.15s)
    shared_test.go:41: executing ../../bin/trivial (trivial executable) failed signal: segmentation fault (core dumped):
--- FAIL: TestDivisionExecutable (3.22s)
    shared_test.go:41: executing ../../bin/division (division executable) failed signal: segmentation fault (core dumped):
--- FAIL: TestCgoExecutable (6.45s)
    shared_test.go:41: executing ../../bin/execgo (cgo executable) failed signal: segmentation fault (core dumped):
--- FAIL: TestGopathShlib (10.57s)
    shared_test.go:41: executing ../../bin/exe (executable linked to GOPATH library) failed signal: segmentation fault (core dumped):
--- FAIL: TestTwoGopathShlibs (11.90s)
    shared_test.go:41: executing ../../bin/exe2 (executable linked to GOPATH library) failed signal: segmentation fault (core dumped):
--- FAIL: TestThreeGopathShlibs (13.34s)
    shared_test.go:41: executing ../../bin/exe3 (executable linked to GOPATH library) failed signal: segmentation fault (core dumped):
--- FAIL: TestABIChecking (6.29s)
    shared_test.go:885: exe failed, but without line "abi mismatch detected between the executable and lib"; got output:
--- FAIL: TestImplicitInclusion (3.27s)
    shared_test.go:41: executing ../../bin/implicitcmd (running executable linked against library that contains same package as it) failed signal: segmentation fault (core dumped):
--- FAIL: TestInterface (4.50s)
    shared_test.go:41: executing ../../bin/iface (running type/itab uniqueness tester) failed signal: segmentation fault (core dumped):
--- FAIL: TestGlobal (3.58s)
    shared_test.go:41: executing ../../bin/global (global executable) failed signal: segmentation fault (core dumped):
shared_test.go:72: executing go test -installsuffix=6129484611666145821 -linkshared -test.short sync/atomic failed exit status 1:
exit status 1
FAIL	misc/cgo/testshared	111.018s
2019/05/03 10:43:35 Failed: exit status 1
run: go list -gcflags=all= -ldflags=all= -f={{if .Stale}}	STALE {{.ImportPath}}: {{.StaleReason}}{{end}} std
run: go list -gcflags=all= -ldflags=all= -f={{if .Stale}}	STALE {{.ImportPath}}: {{.StaleReason}}{{end}} std DONE
@ianlancetaylor ianlancetaylor changed the title cgo testshared segfults on linux/arm64 Fedora 29 misc/cgo/testshared: segfults on linux/arm64 Fedora 29 May 3, 2019
@ianlancetaylor
Copy link
Contributor

Odd. The builder isn't reporting any problems.

Can you reproduce the problem cd misc/cgo/testshared; go test -v? What is the output?

@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker labels May 3, 2019
@ianlancetaylor ianlancetaylor added this to the Go1.13 milestone May 3, 2019
@jcajka jcajka changed the title misc/cgo/testshared: segfults on linux/arm64 Fedora 29 misc/cgo/testshared: segfaults on linux/arm64 Fedora 29 May 6, 2019
@jcajka
Copy link
Contributor Author

jcajka commented May 7, 2019 •
edited

Yeah, it looks odd. Possibly something wrong with the binaries/runtime. Build/test log looks sane to me, although failing. This unfortunately flew under my radar for some time :(.
Output of cd misc/cgo/testshared; go test -c -v; ./testshared.test -test.v
About to dig in to the binaries built during the test.

For the record what are the versions of binutils, glibc and gcc on builders? To rule out that angle.

@ianlancetaylor
Copy link
Contributor

CC @bradfitz: see question about tools versions on builders. Thanks.

@ianlancetaylor
Copy link
Contributor

Actually, never mind, I can use gomote to find the tool versions.

We're using gcc 4.9.2, binutils 2.25, glibc 2.19-18. So quite a bit older than the tools you are using. We are running Debian jessie.

@ianlancetaylor
Copy link
Contributor

Thanks for sending the log output. Unfortunately I agree that it all looks normal. My first guess would of course be that the newer tools are causing a problem somehow, but I don't know why that would be. I think we'll need some way to reproduce the problem ourselves.

@bradfitz
Copy link
Contributor

bradfitz commented May 7, 2019

I can add a Fedora builder, but in the meantime (still commuting) I just updated our debian-sid builder. It hadn't been updated in quite some time, and it gcc was pretty old.

From 7.3.0 to 8.3.0:

bradfitz@go:~/src/golang.org/x/build/env/linux-x86-sid$ docker run -ti gcr.io/symbolic-datum-552/linux-x86-sid:latest /bin/bash
root@759b9aca4e34:/# gcc --version
gcc (Debian 7.3.0-18) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

bradfitz@go:~/src/golang.org/x/build/env/linux-x86-sid$ docker run -ti golang/linux-x86-sid /bin/bash
root@a6236231b605:/# gcc --version
gcc (Debian 8.3.0-7) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Just deployed now.

@gopherbot
Copy link

Change https://golang.org/cl/175737 mentions this issue: env/linux-x86-sid: update our sid builder

@jcajka
Copy link
Contributor Author

jcajka commented May 7, 2019

@ianlancetaylor Just did quick poke on it with gdb and the crash seems to happen during executable startup prior main function is invoked.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x0000ffffbf2c8d24 in __libc_start_main () from /lib64/libc.so.6
#2  0x0000000000400a34 in _start ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Intriguing is that the shared pie tests pass just fine.

To note I will continue digging in to this next week as I will be mostly offline for next 7 days.

@ianlancetaylor
Copy link
Contributor

@bradfitz Note that this bug is for arm64. I can't tell whether you updated the arm64 builder or not.

gopherbot pushed a commit to golang/build that referenced this issue May 7, 2019
@bradfitz
env/linux-x86-sid: update our sid builder
310754d 
Updates golang/go#31823

Change-Id: I96d83cf7a9994c8ed6d0ea58af038577d48ae737
Reviewed-on: https://go-review.googlesource.com/c/build/+/175737
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@bradfitz
Copy link
Contributor

bradfitz commented May 7, 2019

Note that this bug is for arm64. I can't tell whether you updated the arm64 builder or not.

I just did linux-amd64-sid out of hope that updated tools would let us reproduce it alone. And we try to update the sid builder regularly anyway, so doesn't hurt.

But sadly that didn't catch it. We do have enough arm64 capacity that we could run a Fedora arm64 Docker container on it, assuming that'd be enough and it's not a kernel issue. That's a bit more tedious so I likely won't get to it right away.

@bradfitz
Copy link
Contributor

bradfitz commented May 8, 2019

I also tried Fedora 30 (also amd64) on a whim but it passed fine. I didn't try Fedora 29 on amd64, though.

@jcajka
Copy link
Contributor Author

jcajka commented May 21, 2019

More digging in, this is actually dupe of #28334 and it has been/is binutils/gold issue https://sourceware.org/bugzilla/show_bug.cgi?id=23870 fixed on the trunk.
So closing. Thanks to nickc for quick backport of the patch for all Fedora releases.

@jcajka jcajka closed this as completed May 21, 2019
@shawndx
Copy link
Contributor

shawndx commented Oct 22, 2019 •
edited

Hi,

Got the same issue on arm64, seems that it's caused (one of the reasons I think) by 'default pie' not being enabled, the issue could be reproduced by clang (6.0.0-1ubuntu2) and gcc without "--enable-default-pie" in my environment.

I have a preliminary fix which introduces '-extldflags=-pie' in gc-toolchain's ld part under a few conditions (arm64, building linkshared, etc.), before finalizing the fix I'd like to ask for the expert's input if adding -extldflags is an option? Thanks.

@ianlancetaylor
Copy link
Contributor

@shawn-xdji This issue was closed several months ago, so this is not the best place to discuss possible changes. Please open a new issue or send a message to golang-dev. Thanks.

@golang golang locked and limited conversation to collaborators Oct 21, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
5 participants
@bradfitz @ianlancetaylor @jcajka @gopherbot @shawndx