Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: relocation target memcpy not defined for ABI0 (but is defined for ABI0) #33492

Closed
prestonvanloon opened this issue Aug 6, 2019 · 11 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@prestonvanloon
Copy link

prestonvanloon commented Aug 6, 2019

What version of Go are you using (go version)?

go version go1.12.7 linux/amd64                                                 
GOARCH="amd64"                                                                  
GOBIN=""                                                                        
GOCACHE="off"                                                                   
GOEXE=""                                                                        
GOFLAGS=""                                                                      
GOHOSTARCH="amd64"                                                              
GOHOSTOS="linux"                                                                
GOOS="linux"                                                                    
GOPATH=""                                                                       
GOPROXY=""                                                                      
GORACE=""                                                                       
GOROOT="/home/preston/.cache/bazel/_bazel_preston/2bf51e8690558a83a0a0514ab422fd4b/external/go_sdk"
GOTMPDIR=""                                                                     
GOTOOLDIR="/home/preston/.cache/bazel/_bazel_preston/2bf51e8690558a83a0a0514ab422fd4b/external/go_sdk/pkg/tool/linux_amd64"
GCCGO="gccgo"                                                                   
CC="gcc"                                                                        
CXX="g++"                                                                       
CGO_ENABLED="1"                                                                 
GOMOD=""                                                                        
CGO_CFLAGS="-g -O2"                                                             
CGO_CPPFLAGS=""                                                                 
CGO_CXXFLAGS="-g -O2"                                                           
CGO_FFLAGS="-g -O2"                                                             
CGO_LDFLAGS="-g -O2"                                                            
PKG_CONFIG="pkg-config"                                                         
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build969961047=/tmp/go-build -gno-record-gcc-switches"

Does this issue reproduce with the latest release?

I didn't try. 1.12.7 is the latest. :)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

What did you do?

Link rust FFI archives with go.

What did you expect to see?

Something more meaningful.

What did you see instead?

relocation target memset not defined for ABI0 (but is defined for ABI0)
relocation target memcpy not defined for ABI0 (but is defined for ABI0)

ERROR: /home/preston/go-packages/src/github.com/prysmaticlabs/prysm/shared/bls/BUILD.bazel:85:1: GoLink shared/bls/linux_amd64_stripped/rustgo_lib_benchmark_test failed (Exit 1) builder failed: error executing command
  (cd /home/preston/.cache/bazel/_bazel_preston/2bf51e8690558a83a0a0514ab422fd4b/sandbox/linux-sandbox/2661/execroot/__main__ && \
  exec env - \
    CGO_ENABLED=1 \
    GOARCH=amd64 \
    GOOS=linux \
    GOROOT=external/go_sdk \
    GOROOT_FINAL=GOROOT \
    PATH=/usr/bin:/bin \
  bazel-out/host/bin/external/go_sdk/builder link -sdk external/go_sdk -installsuffix linux_amd64 -arc '//shared/bls:rustgo_lib=github.com/prysmaticlabs/prysm/shared/bls=bazel-out/k8-fastbuild/bin/shared/bls/linux_amd64_stripped/rustgo_lib%/github.com/prysmaticlabs/prysm/shared/bls.a' -arc '//shared/bls:rustgo_lib_benchmark_test=shared/bls/rustgo_lib_benchmark_test_test=bazel-out/k8-fastbuild/bin/shared/bls/linux_amd64_stripped/rustgo_lib_benchmark_test%/shared/bls/rustgo_lib_benchmark_test_test.a' -arc '//shared/bls:rustgo_lib_benchmark_test=shared/bls/rustgo_lib_benchmark_test=bazel-out/k8-fastbuild/bin/shared/bls/linux_amd64_stripped/rustgo_lib_benchmark_test%/shared/bls/rustgo_lib_benchmark_test.a' -package_list bazel-out/host/bin/external/go_sdk/packages.txt -o bazel-out/k8-fastbuild/bin/shared/bls/linux_amd64_stripped/rustgo_lib_benchmark_test -main bazel-out/k8-fastbuild/bin/shared/bls/linux_amd64_stripped/rustgo_lib_benchmark_test%/testmain.a -p testmain -- -extld /usr/bin/gcc '-buildid=redacted' -w -extldflags '-fuse-ld=gold -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/usr/bin -pass-exit-codes -lm')
Execution platform: @bazel_tools//platforms:host_platform

Use --sandbox_debug to see verbose messages from the sandbox
github.com/prysmaticlabs/prysm/shared/bls(.text._ZN8amcl_lib4rand4RAND3new17h4543d242f70bcfc2E): relocation target memset not defined for ABI0 (but is defined for ABI0)
github.com/prysmaticlabs/prysm/shared/bls(.text._ZN8amcl_lib4rand4RAND3new17h4543d242f70bcfc2E): relocation target memcpy not defined for ABI0 (but is defined for ABI0)
link: error running subcommand: exit status 2
@dmitshur dmitshur changed the title link: relocation target memcpy not defined for ABI0 (but is defined for ABI0) cmd/link: relocation target memcpy not defined for ABI0 (but is defined for ABI0) Aug 6, 2019
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 6, 2019
@dmitshur dmitshur added this to the Go1.14 milestone Aug 6, 2019
@dmitshur
Copy link
Contributor

dmitshur commented Aug 6, 2019

Does this issue reproduce with the latest release?

I didn't try.

You did use Go 1.12.7, which is the latest release.

/cc @cherrymui FYI per owners.

@ianlancetaylor
Copy link
Contributor

Can you give precise instructions for how to reproduce the problem? Thanks.

@dmitshur dmitshur added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 6, 2019
@prestonvanloon
Copy link
Author

prestonvanloon commented Aug 6, 2019

I compiled a rust archive with rustc and llvm optimization flag --codegen=lto. Trying to link that with go was causing the issue.

When I turned on optimizations --codegen=opt-level=3 then the problem went away. It was adding references to memcpy or memset that weren't used, I guess. I filed the issue because the error message is confusing.

If you need a minimal code example, I can put one together.

Edit: actually it came back after adding more logic. Not sure what is going on with missing memset, memcpy.

@randall77
Copy link
Contributor

If you need a minimal code example, I can put one together.

Yes, please do.

prestonvanloon added a commit to prestonvanloon/go-issue-33492 that referenced this issue Aug 6, 2019
@prestonvanloon
Copy link
Author

Here's a reproduction: https://github.com/prestonvanloon/go-issue-33492

It's as minimal as I could make it.

@randall77
Copy link
Contributor

Hmmm.... Calling into C directly from Go assembly is not a supported mechanism. Use cgo instead.

Allocating a large frame in assembly, rewinding the stack frame, then using that empty stack frame as a C stack is doubly not supported.

That said, the problem you're running into has nothing to do with that. I'm not sure why you're getting that error. It sounds like the Go linker is somehow trying to link the C code and getting confused by it. Probably not worth digging into it given the above, though. If you switch to using cgo and still run into this problem, feel free to reopen.

@randall77
Copy link
Contributor

@FiloSottile

@FiloSottile
Copy link
Contributor

I need to put a flashier disclaimer on https://blog.filippo.io/rustgo. It was an experiment and a learning exercise, not something reproducible.

(Your issue though looks like you need to link a libc? Or maybe didn't succeed in making the Rust object no_std? If you want to continue down this path, I'd try forcing the external linker, and then debugging it as you would debug linking anything without the go build toolchain support. But again, this is not supported and it will eventually break and won't be fixed on our side.)

@prestonvanloon
Copy link
Author

The main idea here was to avoid the cgo performance overhead, if it were possible. @FiloSottile's article looked promising, albeit a bit dated. I suspected I might hit a dead end here, but wanted to try it anyway.

I'll give the cgo approach a try. ~70ns overhead might be tolerable for my use case. Thanks for looking into it. I'll be sure to post a follow up here.

@aykevl
Copy link

aykevl commented Apr 30, 2020

As a data point, I've hit this same issue in CI. If I find anything reproducable, I'll add it here. This is the error I got:

.dynsym: missing Go type information for global symbol: size 72
os/user(.text): relocation target malloc not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target free not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target getgrgid_r not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target getgrnam_r not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target getpwnam_r not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target getpwuid_r not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target realloc not defined for ABI0 (but is defined for ABI0)
os/user(.text): relocation target sysconf not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target stderr not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target fwrite not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target vfprintf not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target fputc not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target abort not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target pthread_mutex_lock not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target pthread_cond_wait not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target pthread_mutex_unlock not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target __stack_chk_fail not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target pthread_cond_broadcast not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target pthread_create not defined for ABI0 (but is defined for ABI0)
runtime/cgo(.text): relocation target nanosleep not defined for ABI0 (but is defined for ABI0)
/usr/local/go/pkg/tool/linux_amd64/link: too many errors

What makes this especially weird is the following:

  1. It only happens with go test, not when normally importing the package.
  2. The erroring package does not use CGo directly (only through an intermediate package, which again works just fine with a regular build).

Also, this specifically happens with a CI Docker image.

@ianlancetaylor
Copy link
Contributor

@aykevl This issue is closed. If you do have a reproducible failure, please open a new issue with details. Thanks.

@golang golang locked and limited conversation to collaborators Apr 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

7 participants