Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: value of -buildmode in buildinfo can differ from what is used in build ID #63559

Open
capnspacehook opened this issue Oct 15, 2023 · 3 comments
Open

cmd/go: value of -buildmode in buildinfo can differ from what is used in build ID #63559

capnspacehook opened this issue Oct 15, 2023 · 3 comments
Labels
GoCommand cmd/go help wanted NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Backlog

Comments

@capnspacehook
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.21.3 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/capnspacehook/.cache/go-build'
GOENV='/home/capnspacehook/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/capnspacehook/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/capnspacehook/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.1'
GCCGO='gccgo'
GOAMD64='v3'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD=''
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3246179384=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Built two binaries from identical source code, passing -buildmode=default to the first and -buildmode=exe to the second.

What did you expect to see?

Both binaries should be identical and have the same hash.

What did you see instead?

The binaries hashes differed, though using xbindiff showed that the build ID was the only difference between them. It seems that the buildmode that is actually used is included in binaries build information (exe in both cases), but the exact value of -buildmode as passed to go build is used when creating the build ID. To make reproducing binaries easier I think the same buildmode value that is used in the embedded build information should be used when creating the build ID.

Currently it can be very difficult to tell what value of -buildmode was passed at build time when you only have a Go binary and no information on its build process. This change would make it easier to reproduce Go binaries using the embedded build information as a guide.
@capnspacehook capnspacehook changed the title affected/package: cmd/go: value of -buildmode in buildinfo can differ from what is used in build ID Oct 15, 2023
@bcmills bcmills added GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. help wanted labels Oct 17, 2023
@bcmills
Copy link
Contributor

bcmills commented Oct 17, 2023

The line that puts this data in the cache ID is here:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/work/exec.go;l=1348;drc=ee8b8e988c47a400d035891539a045031854b391

And the logic for the embedded buildinfo is here:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/load/pkg.go;l=2371-2379;drc=5873bd1d7e87e0686390d71a840a44e2e2c3e11d

That said, the embedded logic seems off too: on some platforms, default is pie rather than exe:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/work/init.go;l=231-238;drc=5873bd1d7e87e0686390d71a840a44e2e2c3e11d

And the logic for whether to set codegenArg to -fPIE or -shared or leave it unset is different between the pie and default cases — that's probably a bug?

Either way, this needs a careful test for platforms that default to PIE executables.

@quantonganh
Copy link
Contributor

It seems that the buildmode that is actually used is included in binaries build information (exe in both cases)

If a binary is built using -buildmode=default:

$ go build -buildmode=default -o main.default

Should we record the -buildmode in the build info as it is instead of exe?

$ go version -m main.default
main.default: devel go1.22-777540cd23 Tue Nov 14 08:20:14 2023 +0700
        path    m
        mod     m       (devel)
        build   -buildmode=exe
        build   -compiler=gc
        build   CGO_ENABLED=1
        build   CGO_CFLAGS=
        build   CGO_CPPFLAGS=
        build   CGO_CXXFLAGS=
        build   CGO_LDFLAGS=
        build   GOARCH=arm64
        build   GOOS=darwin

@quantonganh
Copy link
Contributor

When -buildmode=default is used on platforms that default to PIE, codegenArg will be set to -shared:

go/src/cmd/go/internal/work/init.go

Line 236 in c7ea996

codegenArg = "-shared"

This change causes forcedGcflags to be altered:

go/src/cmd/go/internal/work/init.go

Line 201 in 5993251

forcedGcflags = append([]string{codegenArg}, forcedGcflags...)

subsequently affecting the build id:

go/src/cmd/go/internal/work/exec.go

Line 212 in 5993251

fmt.Fprintf(h, "compile %s %q %q\n", b.toolID("compile"), forcedGcflags, p.Internal.Gcflags) 

$ go tool buildid main.default
sXYlCmiozkAhsK7gKdc1/gNrES6_ME4j4MT5LqwVJ/TJYC4wzic3MbCcIJp9vw/OXE-kHIiAcvetgY7Ag7H

$ go tool buildid main.exe
i2tRUqE42w9e3W5Tm5Oi/oLoaliHU9uVRiFRcb4kx/rcVyJqY-QPyHlugRPly_/Q1mjqnn3Ybz1xs_Efesg

I'm wondering if this behavior aligns with the expected functionality.

@bcmills bcmills added this to the Backlog milestone Jan 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GoCommand cmd/go help wanted NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
3 participants
@quantonganh @bcmills @capnspacehook