Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/link: ABI hash for package in shared library should not include file path of dependency #30531

Closed
eternal-flame-AD opened this issue Mar 2, 2019 · 2 comments
Closed

cmd/link: ABI hash for package in shared library should not include file path of dependency #30531

eternal-flame-AD opened this issue Mar 2, 2019 · 2 comments
Labels
FrozenDueToAge

Comments

@eternal-flame-AD
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.12 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ef/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ef/code/go/"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build320733553=/tmp/go-build -gno-record-gcc-switches"

What did you do?

  1. Write a main package with GO111MODULE, which opens another package as a plugin.
  2. Write another main package with GO111MODULE and a common dependency (the version of the dependency is in equal to that of the first package), which is built as plugin and loaded by the previous package.
  3. Build both packages by go build [-buildmode=plugin] and test the compatibility between them.
  4. Build both packages by go mod vendor && go build -mod=vendor [-buildmode=plugin] and test the compatibility between them.

What did you expect to see?

In both cases the program runs successfully.

What did you see instead?

The package built by go mod vendor is rejected due to different ABI hashes of the common dependency. I dug into this and found that the difference in the ABI hash was caused by that the path to the package on the filesystem was recorded and hashed, as go mod vendor vendors the common depencency into different directories, the final ABI hash is different.
@eternal-flame-AD eternal-flame-AD changed the title cmd/link: ABI hash for package in shared library should not include cmd/link: ABI hash for package in shared library should not include file path of dependency Mar 2, 2019
@bauerm97
Copy link

bauerm97 commented Mar 4, 2019

I think this is related to #16860 , perhaps even a duplicate of the issue. It looks like there are plans in place for Go 1.13 to run go build in a "reproducible" mode, stripping the prefix of the source directory from the final binary. In the mean time, you can use this magic incantation of go build to fix this:

go build -gcflags=all=-trimpath=$(PREFIX) -asmflags=all=-trimpath=$(PREFIX) $(PACKAGE)

That's how we are solving this issue in the meantime while we wait for Go 1.13 to provide a fix. You can see that it works in this example:

mibauer@ubuntu:~/modules-plugins/main$ pwd
/home/mibauer/modules-plugins/main
mibauer@ubuntu:~/modules-plugins/main$ go clean -cache
mibauer@ubuntu:~/modules-plugins/main$ go build -buildmode=plugin -gcflags=all=-trimpath=/home/mibauer/modules-plugins -asmflags=all=-trimpath=/home/mibauer/modules-plugins p1
mibauer@ubuntu:~/modules-plugins/main$ go build -buildmode=plugin -gcflags=all=-trimpath=/home/mibauer/modules-plugins -asmflags=all=-trimpath=/home/mibauer/modules-plugins p2
mibauer@ubuntu:~/modules-plugins/main$ ls
build-vx  dumpdep  go.mod  go.sum  main.go  p1.so  p2.so  pkg  tools  vendor
mibauer@ubuntu:~/modules-plugins/main$ cp p1.so ~/modules-plugins-path2/main/
mibauer@ubuntu:~/modules-plugins/main$ cp p2.so ~/modules-plugins-path2/main/
mibauer@ubuntu:~/modules-plugins/main$ cd ~/modules-plugins-path2/main/
mibauer@ubuntu:~/modules-plugins-path2/main$ go build .
mibauer@ubuntu:~/modules-plugins-path2/main$ ./main 
panic: plugin.Open("p1"): plugin was built with a different version of package gopkg.in/yaml.v2

goroutine 1 [running]:
main.main()
    /home/mibauer/modules-plugins-path2/main/main.go:13 +0x229
mibauer@ubuntu:~/modules-plugins-path2/main$ rm main
mibauer@ubuntu:~/modules-plugins-path2/main$ go build -gcflags=all=-trimpath=/home/mibauer/modules-plugins-path2 -asmflags=all=-trimpath=/home/mibauer/modules-plugins-path2 .
mibauer@ubuntu:~/modules-plugins-path2/main$ ./main 
p1: Type: yaml.Encoder
p2: Type: yaml.Encoder
yaml.Encoder
mibauer@ubuntu:~/modules-plugins-path2/main$

@eternal-flame-AD
Copy link
Author

Thanks very much for the workaround:)

I took a look at the issue you mentioned, it was talking about build path being included in the linking stage while I was talking about in the compiling stage. Never mind, I think the "reproducible" build mode would solve both:)

mem pushed a commit to mem/singularity that referenced this issue Apr 5, 2019
Do not pass -mod=vendor when building singularity
20edf17 
This interferes with plugin loading. See:

- golang/go#31278
- golang/go#30531
- golang/go#28983

Signed-off-by: Marcelo E. Magallon <marcelo@sylabs.io>
@mem mem mentioned this issue Apr 5, 2019
Closed
@golang golang locked and limited conversation to collaborators Mar 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@julieqiu @gopherbot @bauerm97 @eternal-flame-AD