Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: repeatable builds with _cgo_gotypes.go #47256

Closed
kpcyrd opened this issue Jul 16, 2021 · 5 comments
Closed

cmd/go: repeatable builds with _cgo_gotypes.go #47256

kpcyrd opened this issue Jul 16, 2021 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@kpcyrd
Copy link

kpcyrd commented Jul 16, 2021

What version of Go are you using (go version)?

$ go version
1.16.5

Does this issue reproduce with the latest release?

Yes.

What did you do?

We're trying to reproduce all packages over at reproducible.archlinux.org and noticed multiple packages fail due to _cgo_gotypes.go path differences.

What did you expect to see?

Packages are reproducible/the build is repeatable.

What did you see instead?

There are full paths included in the binary, this is usually not a problem in Arch Linux but for _cgo_gotypes.go files this path contains a random temporary directory name.

go-ipfs (built with 1.16.5): https://reproducible.archlinux.org/api/v0/builds/137190/diffoscope
wallutils (built with 1.16): https://reproducible.archlinux.org/api/v0/builds/120436/diffoscope

The differences look like this:

│ │ │ @@ -181581,15 +181581,15 @@
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/ipfs/go-ipfs-chunker/buzhash.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/ipfs/go-ipfs-chunker/parse.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/ipfs/go-ipfs-chunker/splitting.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/ipfs/go-ipfs-chunker/rabin.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/crackcomm/go-gitignore/ignore.go
│ │ │  go/src/os/user/lookup.go
│ │ │  go/src/os/user/user.go
│ │ │ -/tmp/go-build4137611863/b138/_cgo_gotypes.go
│ │ │ +/tmp/go-build2479175138/b138/_cgo_gotypes.go
│ │ │  go/src/os/user/cgo_lookup_unix.go
│ │ │  go/src/os/user/getgrouplist_unix.go
│ │ │  go/src/os/user/listgroups_unix.go
│ │ │  go/src/archive/tar/common.go
│ │ │  go/src/archive/tar/format.go
│ │ │  go/src/archive/tar/reader.go
│ │ │  go/src/archive/tar/stat_unix.go
│ │ │ @@ -181777,15 +181777,15 @@
│ │ │  /build/go-ipfs/src/go-ipfs/core/coreapi/path.go
│ │ │  /build/go-ipfs/src/go-ipfs/core/coreapi/pin.go
│ │ │  /build/go-ipfs/src/go-ipfs/core/coreapi/pubsub.go
│ │ │  /build/go-ipfs/src/go-ipfs/core/coreapi/swarm.go
│ │ │  /build/go-ipfs/src/go-ipfs/core/coreapi/unixfs.go
│ │ │  go/src/plugin/plugin.go
│ │ │  go/src/plugin/plugin_dlopen.go
│ │ │ -/tmp/go-build4137611863/b666/_cgo_gotypes.go
│ │ │ +/tmp/go-build2479175138/b666/_cgo_gotypes.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/ipfs/go-datastore/mount/mount.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/github.com/ipfs/go-ds-measure/measure.go
│ │ │  /build/go-ipfs/src/go-ipfs/repo/common/common.go
│ │ │  /build/go-ipfs/src/go-ipfs/thirdparty/dir/dir.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/go4.org/lock/lock.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/go4.org/lock/lock_sigzero.go
│ │ │  /build/go-ipfs/src/go-ipfs/vendor/go4.org/lock/lock_unix.go

This issue might be similar to #19964.

@cherrymui cherrymui changed the title cmd/cgo: repeatable builds with _cgo_gotypes.go cmd/go: repeatable builds with _cgo_gotypes.go Jul 16, 2021
@cherrymui cherrymui added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 16, 2021
@cherrymui cherrymui added this to the Backlog milestone Jul 16, 2021
@cherrymui
Copy link
Member

@seankhliao
Copy link
Member

I believe this is an issue with the specific build commands used, https://github.com/ipfs/go-ipfs/blob/master/mk/golang.mk#L15 and https://github.com/archlinux/svntogit-community/blob/packages/wallutils/trunk/PKGBUILD#L18

without those options, the embedded path is just _cgo_gotypes.go

@bcmills
Copy link
Contributor

bcmills commented Jul 16, 2021

For reproducible builds with cmd/go you should pass the -trimpath flag to cmd/go itself, not set the low-level flags for the underlying tools individually. (The low-level flags may work in some cases, but that's not really a supported configuration.)

If that higher-level flag doesn't work for some reason, we should address that directly.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jul 16, 2021
@jayconrod
Copy link
Contributor

Related, maybe the root cause: #36072. I don't think -trimpath works in cgo generated code. It should.

@kpcyrd
Copy link
Author

kpcyrd commented Jul 19, 2021

I'm not sure about the other issue, but our original problem was solved by replacing -gcflags and -asmflags with just -trimpath, so closing this one.

Thanks everybody!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

6 participants