Skip to content

cmd/link: intermittently produces segfaulting executables with binutils 2.27-44 #51611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
pjgjordaan opened this issue Mar 11, 2022 · 10 comments
Closed

cmd/link: intermittently produces segfaulting executables with binutils 2.27-44 #51611

pjgjordaan opened this issue Mar 11, 2022 · 10 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Backlog

Comments

@pjgjordaan
Copy link

pjgjordaan commented Mar 11, 2022 •
edited
Loading

What version of Go are you using (go version)?

$ go version
go version go1.17.8 linux/amd64

We build go from source ourselves using this Makefile:

GO_BUILD_VERSION=1.17.8

clonego:
	git clone https://go.googlesource.com/go

checkoutversion:
	(cd go && git checkout go${GO_BUILD_VERSION} )

buildgo:
	(cd go/src && ./make.bash )

targz:
	tar -czf go-${GO_BUILD_VERSION}-linux-amd64.tar.gz ./go

buildall: clonego checkoutversion buildgo targz

clean:
	rm -rf ./go
	rm *tar.gz

.PHONY: clonego checkoutversion buildgo targz clean

.DEFAULT_GOAL:=buildall

The reason for building go ourselves is that from go 1.16 our version of binutils no longer worked with the pre-compiled go release. We encountered the problem described here #43996
Our version of binutils:

$ yum list  binutils
Installed Packages
binutils.x86_64                                                                                     2.27-44.base.fc20                                                                                      @/binutils-2.27-44.base.fc20.x86_64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/pieter/dev/myrepo/bin"
GOCACHE="/home/pieter/.cache/go-build"
GOENV="/home/pieter/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/pieter/go/pkg/mod"
GONOPROXY="*.mydomain.co.za"
GONOSUMDB="*.mydomain.co.za"
GOOS="linux"
GOPATH="/home/pieter/go"
GOPRIVATE="*.mydomain.co.za"
GOPROXY="http://artifactory1.mydomain.co.za:8081/artifactory/go-modules"
GOROOT="/home/pieter/dev/go"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/home/pieter/dev/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.8"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/pieter/dev/myrepo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build99898503=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.17.8 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.17.8
uname -sr: Linux 3.10.0-693.fc20.x86_64
/lib64/libc.so.6: GNU C Library (GNU libc) stable release version 2.18, by Roland McGrath et al.
gdb --version: GNU gdb (GDB) Fedora 7.7.1-21.fc20

What did you do?

I built 1000 'empty' go executables with: @go build -o ./bin -compiler=gc ./vt/tools/dummies/...
The only code in these programs is:

package main

func main(){}

All of the programs are in the ./vt/tools/dummies folder and are named dummy0 ... dummy999.
After compiling them successfully 148 times, on the 149th compilation on of the binaries cannot be executed and segfaults:

[pieter@devbox2 next]$ bin/dummy191
Segmentation fault

The go build command itself exits cleanly.

What did you expect to see?

go build to produce valid executables, and if it doesn't, it should fail.

What did you see instead?

go build exits cleanly but produced an executable that segfaults.

I compared the segfaulting binary to a valid binary in Ghidra. The segfaulting one is on the left (dummy191.sf)

Screenshot 2022-03-10 at 13 16 05

Screenshot 2022-03-10 at 13 12 21

Screenshot 2022-03-10 at 13 14 42

Screenshot 2022-03-10 at 13 12 43

Screenshot 2022-03-10 at 13 13 49

Screenshot 2022-03-10 at 13 12 55

Screenshot 2022-03-10 at 13 13 39

Screenshot 2022-03-10 at 13 13 08
@mengzhuo mengzhuo changed the title cmd/go go build intermittently produces in segfaulting executables cmd/go: go build intermittently produces in segfaulting executables Mar 11, 2022
@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 11, 2022
@ALTree
Copy link
Member

ALTree commented Mar 11, 2022

Is this specifically a 1.17.8 thing? Or you can reproduce this with other releases?

@tianhendi

This comment was marked as off-topic.

Sign in to view
@pjgjordaan
Copy link
Author

I was also able to reproduce it on 1.17.
I haven't looked further back than that.

@robpike robpike changed the title cmd/go: go build intermittently produces in segfaulting executables cmd/go: go build intermittently produces segfaulting executables Mar 11, 2022
@pjgjordaan
Copy link
Author

pjgjordaan commented Mar 15, 2022 •
edited
Loading

Updated case info with reason for building go from source.
We suspect that the version of binutils (2.27-44) on our OS could be the reason for the segfaults.

@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2022

Do the test programs that you are building have C dependencies?

What happens if you build with CGO_ENABLED=0 and/or -ldflags=all=-linkmode=internal?

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Mar 15, 2022
@bcmills bcmills added this to the Backlog milestone Mar 15, 2022
@pjgjordaan
Copy link
Author

They do not have C dependencies.
I'll test with those flags added.

@pjgjordaan
Copy link
Author

@bcmills I still encountered the seg fault with both CGO_ENABLED=0 and -ldflags=all=-linkmode=internal
I only tested with both set.
The code I am testing is completely dependency free, it is just:

package main

func main(){}

So I would not think that CGO_ENABLED or linkmode=internal should affect it.

@pjgjordaan
Copy link
Author

Is there anything else you need from me?

@bcmills bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 8, 2022
@bcmills bcmills changed the title cmd/go: go build intermittently produces segfaulting executables cmd/link: intermittently produces segfaulting executables with binutils 2.27-44 Apr 8, 2022
@bcmills
Copy link
Contributor

bcmills commented Apr 8, 2022

Are there any differences in the actual commands used to build the executables?

(What differences, if any, do you see if you run go clean -cache, then build the individual binaries with the -x flag passed to go build to get it to dump the commands as it executes them?)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 8, 2022
@gopherbot
Copy link
Contributor

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@bcmills bcmills mentioned this issue Jul 12, 2022
Closed
@golang golang locked and limited conversation to collaborators May 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@pjgjordaan @ALTree @tianhendi @bcmills @gopherbot