How much of the -buildmode=shared -linkshared options should work with ppc64le? I see it is mentioned in the Go 1.6 release notes as being supported for ppc64le but I have not used it before.

I've been trying various ways to use it but can't make it work.

From looking through various documentation I need to make sure I have libstd.so, and possibly libcmd.so. When I try to build using -buildmode=shared I am getting errors like this:

/home/boger/golang/ibm-go1.6/go/pkg/tool/linux_ppc64le/link -o $WORK/lib.so -L $WORK -installsuffix dynlink -buildmode=shared -X k8s.io/kubernetes/pkg/version.buildDate=2016-05-25T18:15:36Z -X k8s.io/kubernetes/pkg/version.gitCommit=025b0172774770f0d5eee3e9604c1a2d938976e0 -X k8s.io/kubernetes/pkg/version.gitTreeState=dirty -X k8s.io/kubernetes/pkg/version.gitVersion=v1.3.0-alpha.4.474+025b0172774770-dirty -X k8s.io/kubernetes/pkg/version.gitMajor=1 -X k8s.io/kubernetes/pkg/version.gitMinor=3+ -v -extldflags -Wl,--print-map -extld=powerpc64le-linux-gnu-gcc runtime/cgo=/home/boger/golang/ibm-go1.6/go/pkg/linux_ppc64le_dynlink/runtime/cgo.a

sync/atomic.StorePointer: sync/atomic.StoreUintptr: not defined
sync/atomic.SwapPointer: sync/atomic.SwapUintptr: not defined
sync/atomic.CompareAndSwapPointer: sync/atomic.CompareAndSwapUintptr: not defined
runtime.sync_atomic_CompareAndSwapUintptr·f: sync/atomic.CompareAndSwapUintptr: not defined
runtime.sync_atomic_StoreUintptr·f: sync/atomic.StoreUintptr: not defined
runtime.sync_atomic_SwapUintptr·f: sync/atomic.SwapUintptr: not defined

I can see that the symbol sync/atomic.StoreUintptr is in my libstd.so, but it is not finding it, unless I need to do something to get it to find it.

It should work, but hasn't been deeply tested. I suspect that it's the -X
arguments that are causing the problems you initially reported.

I don't understand that second link failure, it seems to be trying to link
runtime/cgo only, which makes no real sense. What commands are you running?

On 26 May 2016 at 06:33, Lynn Boger notifications@github.com wrote:

How much of the -buildmode=shared -linkshared options should work with
ppc64le? I see it is mentioned in the Go 1.6 release notes as being
supported for ppc64le but I have not used it before.

I've been trying various ways to use it but can't make it work.

From looking through various documentation I need to make sure I have
libstd.so, and possibly libcmd.so. When I try to build using
-buildmode=shared I am getting errors like this:

/home/boger/golang/ibm-go1.6/go/pkg/tool/linux_ppc64le/link -o
$WORK/lib.so -L $WORK -installsuffix dynlink -buildmode=shared -X
k8s.io/kubernetes/pkg/version.buildDate=2016-05-25T18:15:36Z -X
k8s.io/kubernetes/pkg/version.gitCommit=025b0172774770f0d5eee3e9604c1a2d938976e0
-X k8s.io/kubernetes/pkg/version.gitTreeState=dirty -X
k8s.io/kubernetes/pkg/version.gitVersion=v1.3.0-alpha.4.474+025b0172774770-dirty
-X k8s.io/kubernetes/pkg/version.gitMajor=1 -X
k8s.io/kubernetes/pkg/version.gitMinor=3+ -v -extldflags -Wl,--print-map
-extld=powerpc64le-linux-gnu-gcc
runtime/cgo=/home/boger/golang/ibm-go1.6/go/pkg/linux_ppc64le_dynlink/runtime/cgo.a

sync/atomic.StorePointer: sync/atomic.StoreUintptr: not defined

sync/atomic.SwapPointer: sync/atomic.SwapUintptr: not defined

sync/atomic.CompareAndSwapPointer: sync/atomic.CompareAndSwapUintptr: not
defined

runtime.sync_atomic_CompareAndSwapUintptr·f:
sync/atomic.CompareAndSwapUintptr: not defined

runtime.sync_atomic_StoreUintptr·f: sync/atomic.StoreUintptr: not defined

runtime.sync_atomic_SwapUintptr·f: sync/atomic.SwapUintptr: not defined

I can see that the symbol sync/atomic.StoreUintptr is in my libstd.so, but
it is not finding it, unless I need to do something to get it to find it.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#15770 (comment)

I can't reproduce this (with the go 1.6.2 that's on yakkety):

(yakkety-ppc64el)root@diamond:~/kubernetes# git diff
diff --git a/hack/lib/golang.sh b/hack/lib/golang.sh
index 0ff6e97..2e766e7 100755
--- a/hack/lib/golang.sh
+++ b/hack/lib/golang.sh
@@ -454,7 +454,7 @@ kube::golang::build_binaries_for_platform() {
   else
     # Use go install.
     if [[ "${#nonstatics[@]}" != 0 ]]; then
-      go install "${goflags[@]:+${goflags[@]}}" \
+      go install -buildmode=shared -linkshared
"${goflags[@]:+${goflags[@]}}" \
         -ldflags "${goldflags}" \
         "${nonstatics[@]:+${nonstatics[@]}}"
     fi
(yakkety-ppc64el)root@diamond:~/kubernetes# make
hack/build-go.sh
Go version: go version go1.6.2 linux/ppc64le
+++ [0526 00:04:29] Building the toolchain targets:
    k8s.io/kubernetes/hack/cmd/teststale
+++ [0526 00:04:31] Building go targets for linux/ppc64le:
    cmd/kube-dns
    cmd/kube-proxy
    cmd/kube-apiserver
    cmd/kube-controller-manager
    cmd/kubelet
    cmd/kubemark
    cmd/hyperkube
    federation/cmd/federation-apiserver
    federation/cmd/federation-controller-manager
    plugin/cmd/kube-scheduler
    cmd/kubectl
    cmd/integration
    cmd/gendocs
    cmd/genkubedocs
    cmd/genman
    cmd/genyaml
    cmd/mungedocs
    cmd/genbashcomp
    cmd/genswaggertypedocs
    cmd/linkcheck
    examples/k8petstore/web-server/src
    vendor/github.com/onsi/ginkgo/ginkgo
    test/e2e/e2e.test
    test/e2e_node/e2e_node.test
+++ [0526 00:07:45] Placing binaries
(yakkety-ppc64el)root@diamond:~/kubernetes# echo $?
0

I doubt something between 1.6.1 and 1.6.2 fixed this but who knows.

On 26 May 2016 at 09:35, Michael Hudson-Doyle michael.hudson@canonical.com
wrote:

It should work, but hasn't been deeply tested. I suspect that it's the -X
arguments that are causing the problems you initially reported.

I don't understand that second link failure, it seems to be trying to link
runtime/cgo only, which makes no real sense. What commands are you running?

On 26 May 2016 at 06:33, Lynn Boger notifications@github.com wrote:

How much of the -buildmode=shared -linkshared options should work with
ppc64le? I see it is mentioned in the Go 1.6 release notes as being
supported for ppc64le but I have not used it before.

I've been trying various ways to use it but can't make it work.

From looking through various documentation I need to make sure I have
libstd.so, and possibly libcmd.so. When I try to build using
-buildmode=shared I am getting errors like this:

/home/boger/golang/ibm-go1.6/go/pkg/tool/linux_ppc64le/link -o
$WORK/lib.so -L $WORK -installsuffix dynlink -buildmode=shared -X
k8s.io/kubernetes/pkg/version.buildDate=2016-05-25T18:15:36Z -X
k8s.io/kubernetes/pkg/version.gitCommit=025b0172774770f0d5eee3e9604c1a2d938976e0
-X k8s.io/kubernetes/pkg/version.gitTreeState=dirty -X
k8s.io/kubernetes/pkg/version.gitVersion=v1.3.0-alpha.4.474+025b0172774770-dirty
-X k8s.io/kubernetes/pkg/version.gitMajor=1 -X
k8s.io/kubernetes/pkg/version.gitMinor=3+ -v -extldflags -Wl,--print-map
-extld=powerpc64le-linux-gnu-gcc
runtime/cgo=/home/boger/golang/ibm-go1.6/go/pkg/linux_ppc64le_dynlink/runtime/cgo.a

sync/atomic.StorePointer: sync/atomic.StoreUintptr: not defined

sync/atomic.SwapPointer: sync/atomic.SwapUintptr: not defined

sync/atomic.CompareAndSwapPointer: sync/atomic.CompareAndSwapUintptr: not
defined

runtime.sync_atomic_CompareAndSwapUintptr·f:
sync/atomic.CompareAndSwapUintptr: not defined

runtime.sync_atomic_StoreUintptr·f: sync/atomic.StoreUintptr: not defined

runtime.sync_atomic_SwapUintptr·f: sync/atomic.SwapUintptr: not defined

I can see that the symbol sync/atomic.StoreUintptr is in my libstd.so,
but it is not finding it, unless I need to do something to get it to find
it.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#15770 (comment)

Building kubernetes is new to me so I'm still learning about the makefiles and what should get built. And it seems to behave differently as commits are added to it, so the behavior is not consistent.

If you pull from the latest then you must update the hack/lib/golang.sh file to enable builds for ppc64le.

+++ b/hack/lib/golang.sh
@@ -69,7 +69,7 @@ else
linux/amd64
linux/arm
linux/arm64

• #linux/ppc64le # temporarily disabled due to a linking error
• linux/ppc64le # temporarily disabled due to a linking error
  )

@@ -79,7 +79,7 @@ else
linux/386
linux/arm
linux/arm64

• #linux/ppc64le # temporarily disabled due to a linking error
• linux/ppc64le # temporarily disabled due to a linking error
  darwin/amd64
  darwin/386
  windows/amd64

The original error happens when building the hyperkube binary, and that error was a relocation truncation error due to the text section getting too large for the GNU linker to handle. After the original failure, in some later commits it does build successfully, it just depends on the size that it ends up being or the length for the branch targets. In any case, I was hoping that the use of -buildmode=shared -linkshared would resolve the original issue and allow the build to succeed but maybe that is a wrong assumption. In any case I would like to understand how the buildmode and linkshared options should be used in this case even if it doesn't solve my original problem.

If the build of the hyperkube binary succeeds (without the -buildmode=shared -linkshared options) you should see these:
./_output/local/bin/linux/ppc64le/hyperkube
./_output/local/go/bin/hyperkube

Here's what I tried, and maybe I'm not using the options correctly.

I first did:
go install -buildmode=shared std

In my experiments, when I add -buildmode=shared -linkshared to 'go install' in golang.sh it doesn't build the hyperkube binary so it's not really doing what it needs to do. Not sure if that is expected when using those options.
When I added just -bulidmode=shared, I get the errors I posted earlier, maybe that that is incorrect usage of the options.

Any comments or advice would be helpful.

@mwhudson
I am not able to reproduce the original error. After lots of experimentation I can see that I was misunderstanding how these options should work. It appears that each package that you want to be built as a shared library must be explicitly built with the -buildmode=shared -linkshared options. I was working under the false assumption that the dependent packages would also get built as shared.

I think I have found a real bug with the used of -linkshared and -buildmode and not sure if it is ppc64le only. I can change the title or close this one and open a new one.

If std has been built as a shared library and you try to use -buildmode=shared -linkshared with something that imports golang.org/x/sys/unix, you will get errors like this:

host link: "powerpc64le-linux-gnu-gcc" "-m64" "-gdwarf-2" "-Wl,-znow" "-o" "/tmp/go-build873207612/k8s.io/kubernetes/cmd/kubelet/_obj/exe/a.out" "-rdynamic" "/tmp/go-link-758373105/go.o" "/tmp/go-link-758373105/000000.o" "/tmp/go-link-758373105/000001.o" "/tmp/go-link-758373105/000002.o" "-L/home/boger/golang/go1.6/go/pkg/linux_ppc64le_dynlink/" "-Wl,-rpath=/home/boger/golang/go1.6/go/pkg/linux_ppc64le_dynlink/" "-lstd" "-g" "-O2" "-g" "-O2" "-g" "-O2"
/home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/link: running powerpc64le-linux-gnu-gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-758373105/go.o: In function local.k8s.io/kubernetes/vendor/golang.org/x/sys/unix.Syscall': (.text+0xbb0560): call tosyscall.Syscall' lacks nop, can't restore toc; recompile with -fPIC
/usr/bin/ld: /tmp/go-link-758373105/go.o: In function local.k8s.io/kubernetes/vendor/golang.org/x/sys/unix.Syscall': (.text+0xbb0560): unresolvable R_PPC64_REL24 againstsyscall.Syscall'
/usr/bin/ld: /tmp/go-link-758373105/go.o: In function local.k8s.io/kubernetes/vendor/golang.org/x/sys/unix.Syscall6': (.text+0xbb0570): call tosyscall.Syscall6' lacks nop, can't restore toc; recompile with -fPIC
/usr/bin/ld: /tmp/go-link-758373105/go.o: In function local.k8s.io/kubernetes/vendor/golang.org/x/sys/unix.Syscall6': (.text+0xbb0570): unresolvable R_PPC64_REL24 againstsyscall.Syscall6'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

The instruction it is complaining about is a simple branch instruction. The asm code looks like this:
// Just jump to package syscall's implementation for all these functions.
// The runtime may know about them.

TEXT ·Syscall(SB),NOSPLIT,$0-56
BR syscall·Syscall(SB)

TEXT ·Syscall6(SB),NOSPLIT,$0-80
BR syscall·Syscall6(SB)

TEXT ·RawSyscall(SB),NOSPLIT,$0-56
BR syscall·RawSyscall(SB)

TEXT ·RawSyscall6(SB),NOSPLIT,$0-80
BR syscall·RawSyscall6(SB)

The ppc64le asm is just a branch.

Here is a simple way to reproduce the problem I found. There are actually two problems.

1. If you use the -linkshared option with go get, it does not use linux_ppc64le_dynlink as the install directory.
   go get -u -x -linkshared golang.org/x/sys/unix
   cd .
   git clone https://go.googlesource.com/sys /home/boger/golang.work/src/golang.org/x/sys
   cd /home/boger/golang.work/src/golang.org/x/sys
   git submodule update --init --recursive
   cd /home/boger/golang.work/src/golang.org/x/sys
   git show-ref
   cd /home/boger/golang.work/src/golang.org/x/sys
   git submodule update --init --recursive
   WORK=/tmp/go-build954411439
   mkdir -p $WORK/golang.org/x/sys/unix/_obj/
   mkdir -p $WORK/golang.org/x/sys/
   cd /home/boger/golang.work/src/golang.org/x/sys/unix
   /home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/compile -o $WORK/golang.org/x/sys/unix.a -trimpath $WORK -dynlink -p golang.org/x/sys/unix -installsuffix dynlink -buildid c6471c5554bcf97afeb2aba390b30ab58fbaf41d -D _/home/boger/golang.work/src/golang.org/x/sys/unix -I $WORK -pack -asmhdr $WORK/golang.org/x/sys/unix/_obj/go_asm.h ./bluetooth_linux.go ./constants.go ./env_unix.go ./env_unset.go ./flock.go ./race0.go ./sockcmsg_linux.go ./sockcmsg_unix.go ./str.go ./syscall.go ./syscall_linux.go ./syscall_linux_ppc64x.go ./syscall_unix.go ./zerrors_linux_ppc64le.go ./zsyscall_linux_ppc64le.go ./zsysnum_linux_ppc64le.go ./ztypes_linux_ppc64le.go
   /home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/asm -o $WORK/golang.org/x/sys/unix/_obj/asm.o -trimpath $WORK -I $WORK/golang.org/x/sys/unix/_obj/ -I /home/boger/golang/go1.6/go/pkg/include -D GOOS_linux -D GOARCH_ppc64le -D=GOBUILDMODE_shared=1 -dynlink ./asm.s
   /home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/asm -o $WORK/golang.org/x/sys/unix/_obj/asm_linux_ppc64x.o -trimpath $WORK -I $WORK/golang.org/x/sys/unix/_obj/ -I /home/boger/golang/go1.6/go/pkg/include -D GOOS_linux -D GOARCH_ppc64le -D=GOBUILDMODE_shared=1 -dynlink ./asm_linux_ppc64x.s
   pack r $WORK/golang.org/x/sys/unix.a $WORK/golang.org/x/sys/unix/_obj/asm.o $WORK/golang.org/x/sys/unix/_obj/asm_linux_ppc64x.o # internal
   mkdir -p /home/boger/golang.work/pkg/linux_ppc64le/golang.org/x/sys/
   cp $WORK/golang.org/x/sys/unix.a /home/boger/golang.work/pkg/linux_ppc64le/golang.org/x/sys/unix.a
2. If I use go get -d followed by go install -linkshared, it generates the correct directory:
   go install -x -linkshared golang.org/x/sys/unix
   WORK=/tmp/go-build696097881
   mkdir -p $WORK/golang.org/x/sys/unix/_obj/
   mkdir -p $WORK/golang.org/x/sys/
   cd /home/boger/golang.work/src/golang.org/x/sys/unix
   /home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/compile -o $WORK/golang.org/x/sys/unix.a -trimpath $WORK -dynlink -p golang.org/x/sys/unix -installsuffix dynlink -buildid c6471c5554bcf97afeb2aba390b30ab58fbaf41d -D _/home/boger/golang.work/src/golang.org/x/sys/unix -I $WORK -pack -asmhdr $WORK/golang.org/x/sys/unix/_obj/go_asm.h ./bluetooth_linux.go ./constants.go ./env_unix.go ./env_unset.go ./flock.go ./race0.go ./sockcmsg_linux.go ./sockcmsg_unix.go ./str.go ./syscall.go ./syscall_linux.go ./syscall_linux_ppc64x.go ./syscall_unix.go ./zerrors_linux_ppc64le.go ./zsyscall_linux_ppc64le.go ./zsysnum_linux_ppc64le.go ./ztypes_linux_ppc64le.go
   /home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/asm -o $WORK/golang.org/x/sys/unix/_obj/asm.o -trimpath $WORK -I $WORK/golang.org/x/sys/unix/_obj/ -I /home/boger/golang/go1.6/go/pkg/include -D GOOS_linux -D GOARCH_ppc64le -D=GOBUILDMODE_shared=1 -dynlink ./asm.s
   /home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/asm -o $WORK/golang.org/x/sys/unix/_obj/asm_linux_ppc64x.o -trimpath $WORK -I $WORK/golang.org/x/sys/unix/_obj/ -I /home/boger/golang/go1.6/go/pkg/include -D GOOS_linux -D GOARCH_ppc64le -D=GOBUILDMODE_shared=1 -dynlink ./asm_linux_ppc64x.s
   pack r $WORK/golang.org/x/sys/unix.a $WORK/golang.org/x/sys/unix/_obj/asm.o $WORK/golang.org/x/sys/unix/_obj/asm_linux_ppc64x.o # internal
   mkdir -p /home/boger/golang.work/pkg/linux_ppc64le_dynlink/golang.org/x/sys/
   cp $WORK/golang.org/x/sys/unix.a /home/boger/golang.work/pkg/linux_ppc64le_dynlink/golang.org/x/sys/unix.a
3. But then if I use this testcase with the unix package I see the same errors as above:
   package main

import (
"fmt"
"golang.org/x/sys/unix"
)

func main() {
fmt.Printf("pagesize: %d \n", unix.Getpagesize())
}

I see multiple errors.
go build -linkshared sharedtest.go

command-line-arguments

/home/boger/golang/go1.6/go/pkg/tool/linux_ppc64le/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.(*mmapper).Lock': (.text+0x10c4): call tosync.(_Mutex).Lock' lacks nop, can't restore toc; recompile with -fPIC
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.(_mmapper).Lock': (.text+0x10c4): unresolvable R_PPC64_REL24 against sync.(*Mutex).Lock'
/usr/bin/ld: /tmp/go-link-460846081/go.o: In functionlocal.golang.org/x/sys/unix.(_mmapper).Unlock': (.text+0x1114): call to sync.(_Mutex).Unlock' lacks nop, can't restore toc; recompile with -fPIC
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.(*mmapper).Unlock': (.text+0x1114): unresolvable R_PPC64_REL24 againstsync.(*Mutex).Unlock'
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.Syscall': (.text+0x1128): call tosyscall.Syscall' lacks nop, can't restore toc; recompile with -fPIC
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.Syscall': (.text+0x1128): unresolvable R_PPC64_REL24 againstsyscall.Syscall'
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.Syscall6': (.text+0x1138): call tosyscall.Syscall6' lacks nop, can't restore toc; recompile with -fPIC
/usr/bin/ld: /tmp/go-link-460846081/go.o: In function local.golang.org/x/sys/unix.Syscall6': (.text+0x1138): unresolvable R_PPC64_REL24 againstsyscall.Syscall6'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

same problem happened here in x86_64 with go 1.6.2 too. the problem looked like the same no matter with '-x' option or not.

go install -buildmode=shared -linkshared -s -v -p 4 golang.org/x/text/...

[ 53s] golang.org/x/text/unicode
[ 53s] golang.org/x/text/unicode/rangetable
[ 57s] panic: runtime error: invalid memory address or nil pointer dereference
[ 57s] [signal 0xb code=0x1 addr=0x71 pc=0x414d9b]
[ 57s]
[ 57s] goroutine 20 [running]:
[ 57s] panic(0x93d400, 0xc8200101e0)
[ 57s] /usr/lib64/go/src/runtime/panic.go:481 +0x3e6
[ 57s] main.(_builder).install(0xc820184d90, 0xc8202616c0, 0x0, 0x0)
[ 57s] /usr/lib64/go/src/cmd/go/build.go:1630 +0x5b
[ 57s] main.(_builder).do.func1(0xc8202616c0)
[ 57s] /usr/lib64/go/src/cmd/go/build.go:1260 +0x3b2
[ 57s] main.(_builder).do.func2(0xc8202b8bd0, 0xc820184d90, 0xc8204f6a40)
[ 57s] /usr/lib64/go/src/cmd/go/build.go:1317 +0x156
[ 57s] created by main.(_builder).do
[ 57s] /usr/lib64/go/src/cmd/go/build.go:1323 +0x39e

@laboger, what's the status here? Go1.10?

I'm sorry I didn't see this sooner, sometimes I don't see the notifications for @laboger, not sure why or how to find out.

I just tried this, it still fails. This needs to be compiled as position independent (-shared) but a direct branch to an external target like this is not what the ext linker expects in position independent code. It is the following BR instructions that it is complaining about.

TEXT ·Syscall(SB),NOSPLIT,$0-56
BR syscall·Syscall(SB)

TEXT ·Syscall6(SB),NOSPLIT,$0-80
BR syscall·Syscall6(SB)

TEXT ·RawSyscall(SB),NOSPLIT,$0-56
BR syscall·RawSyscall(SB)

TEXT ·RawSyscall6(SB),NOSPLIT,$0-80
BR syscall·RawSyscall6(SB)

It should probably be done one way for PIC and another for non-PIC.

I looked at this a bit... I think what needs to be done is if there is a BR (branch) with the target too far or in a different name space, then it needs to generate a code sequence like a trampoline, if compiled with -dynlink or -shared.

Obsoleted by #47788