Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: build_trimpath test doesn't correctly test executable reproducibility #35435

Closed
jayconrod opened this issue Nov 7, 2019 · 3 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@jayconrod
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version devel +689f6f77f0 Thu Nov 7 20:14:02 2019 +0000 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/jayconrod/Code/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/jayconrod/Code/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/Code/go/src/cmd/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build459473337=/tmp/go-build -gno-record-gcc-switches -fno-common"

Observed this on both darwin and windows.

What did you do?

I saw this when expanding src/cmd/go/testdata/script/build_trimpath.txt. In short, this test builds a binary in one directory, copies the source and go.mod file to another directory, builds the binary again there, and compares the two executable files. When -trimpath is used, they should be identical.

Here's the modified version of that test I was using.

[short] skip

# A binary built without -trimpath should contain the current workspace
# and GOROOT for debugging and stack traces.
cd a
go build -o $WORK/paths.exe paths.go
exec $WORK/paths.exe $WORK/paths.exe
stdout 'binary contains GOPATH: true'
stdout 'binary contains GOROOT: true'

# A binary built with -trimpath should not contain the current workspace
# or GOROOT.
go build -trimpath -o $WORK/paths.exe paths.go
exec $WORK/paths.exe $WORK/paths.exe
stdout 'binary contains GOPATH: false'
stdout 'binary contains GOROOT: false'

# A binary from an external module built with -trimpath should not contain
# the current workspace or GOROOT.
cd $WORK
env GO111MODULE=on
go get -trimpath rsc.io/fortune
exec $WORK/paths.exe $GOPATH/bin/fortune$GOEXE
stdout 'binary contains GOPATH: false'
stdout 'binary contains GOROOT: false'

# Two binaries built from identical packages in different directories
# should be identical.
mkdir $WORK/_alt/src/a
cp $GOPATH/src/a/go.mod $WORK/_alt/src/a/go.mod
cp $GOPATH/src/a/paths.go $WORK/_alt/src/a/paths.go
cd $WORK/_alt/src/a
go build -trimpath -o paths-alt.exe .
cmp -q $WORK/paths.exe paths-alt.exe

[!exec:gccgo] stop

# Binaries built using gccgo should also be identical to each other.
env GO111MODULE=off # The current released gccgo does not support builds in module mode.
cd $GOPATH/src/a
go build -compiler=gccgo -trimpath -o $WORK/gccgo-GOPATH.exe .

env old_gopath=$GOPATH
env GOPATH=$WORK/_alt
cd $WORK/_alt/src/a
go build -compiler=gccgo -trimpath -o $WORK/gccgo-alt.exe .
cd $WORK
exec $WORK/gccgo-alt.exe $WORK/gccgo-alt.exe
stdout 'binary contains GOPATH: false'
stdout 'binary contains GOROOT: false'
cmp -q gccgo-GOPATH.exe gccgo-alt.exe

-- $GOPATH/src/a/paths.go --
package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
)

func main() {
	exe := os.Args[1]
	data, err := ioutil.ReadFile(exe)
	if err != nil {
		log.Fatal(err)
	}

	gopath := []byte(filepath.ToSlash(os.Getenv("GOPATH")))
	if len(gopath) == 0 {
		log.Fatal("GOPATH not set")
	}
	fmt.Printf("binary contains GOPATH: %v\n", bytes.Contains(data, gopath))

	goroot := []byte(filepath.ToSlash(os.Getenv("GOROOT")))
	if len(goroot) == 0 {
		log.Fatal("GOROOT not set")
	}
	fmt.Printf("binary contains GOROOT: %v\n", bytes.Contains(data, goroot))
}
-- $GOPATH/src/a/go.mod --
module example.com/a

What did you expect to see?

Test should pass.

What did you see instead?

The executable files were not identical. They are the same size and the differences don't seem to be in file paths or build ids. So this doesn't seem to be an exact duplicate of #33772.

@jayconrod jayconrod added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 7, 2019
@jayconrod jayconrod added this to the Go1.14 milestone Nov 7, 2019
@jayconrod jayconrod self-assigned this Nov 7, 2019
@gopherbot
Copy link

Change https://golang.org/cl/208234 mentions this issue: cmd/go: fix and re-enabled build_trimpath test

@jayconrod
Copy link
Contributor Author

For anyone tracking this issue, this was a bug in the test, not in the go command. The test was comparing a binary built from a list of source files with a binary built from a named package. Those aren't necessary (or actually) identical.

@mvdan
Copy link
Member

mvdan commented Nov 23, 2019

@jayconrod perhaps retitle the issue to clarify that the test was buggy? With the current title, I just had a slight scare thinking that Go 1.13.x's -trimpath is broken :)

@jayconrod jayconrod changed the title cmd/go: executable builds not entirely reproducible with -trimpath cmd/go: build_trimpath test doesn't correctly test executable reproducibility Nov 25, 2019
@golang golang locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants