cmd/link: can't build shared time/tzdata #44073

kiap · 2021-02-02T19:08:01Z

What version of Go are you using (`go version`)?

$ go version
1.16rc1

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

Tested on both CentOS 7 and RHEL8

go env Output

$ go env
GO111MODULE=""

GOARCH="amd64"

GOBIN=""

GOCACHE="/usr/src/.cache"

GOENV="/root/.config/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOINSECURE=""

GOMODCACHE="/usr/src/it.dev/pkg/mod"

GONOPROXY=""

GONOSUMDB=""

GOOS="linux"

GOPATH="/usr/src/it.dev"

GOPRIVATE=""

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/usr/src/go"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/usr/src/go/pkg/tool/linux_amd64"

GOVCS=""

GOVERSION="devel +4436114 Fri Jan 29 19:48:48 2021 +0000"

GCCGO="gccgo"

AR="ar"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD="/usr/src/it.dev/src/hello/go.mod"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build561658131=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go install -v -ldflags '-s -w' -gcflags '-trimpath /usr/src/go' -buildmode=shared -linkshared -pkgdir /usr/src/lib time/tzdata

What did you expect to see?

The following shared library being created

/usr/src/lib/libtime-tzdata.so

What did you see instead?

# /tmp/go-build1865403509/b017/libtime-tzdata.so
/usr/src/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-782555625/go.o: relocation R_X86_64_PC32 against undefined symbol `time.registerLoadFromEmbeddedTZData' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

How to solve the issue without shared linking?

go install -v -ldflags '-s -w' -gcflags '-trimpath /usr/src/go' -buildmode=shared -linkshared -pkgdir /usr/src/lib time time/tzdata

However

It creates:
/usr/src/lib/libtime,time-tzdata.so

It is not possible to create time-tzdata as separate shared library, this seems to be the new linker problem.

The text was updated successfully, but these errors were encountered:

seankhliao · 2021-02-02T19:25:21Z

time imports time/tzdata so time/tzdata can't import time which it needs for time.registerLoadFromEmbeddedTZData.
it gets around this with

//go:linkname registerLoadFromEmbeddedTZData time.registerLoadFromEmbeddedTZData

without the import, there's no dependency so time isn't built as part of the go install ... time/tzdata.
a solution is to install time manually first

not sure if there's anything that can be done?

cherrymui · 2021-02-02T19:31:30Z

Yeah, as @seankhliao said, when multiple packages have inter-dependencies, it is not possible to build individual ones as shared libraries. This is also true for the runtime. You'd need to build them together.

kiap · 2021-02-02T19:55:17Z

I think you do not consider I compiled libtime.so before time-tzdata.so

So time-tzdata.so cannot load from shared library libtime.so

seankhliao · 2021-02-02T20:01:58Z

I don't see a problem?

main » rm -rf x

main » go install -v -ldflags '-s -w' -trimpath -buildmode=shared -linkshared -tags timetzdata -pkgdir x time

main » ls x
runtime  libtime.so  runtime.a  time.a  time.shlibname

main » go install -v -ldflags '-s -w' -trimpath -buildmode=shared -linkshared -tags timetzdata -pkgdir x time/tzdata

main » ls x
runtime  time  errors.a  libtime-tzdata.so  libtime.so  runtime.a  sync.a  syscall.a  time.a  time.shlibname

kiap · 2021-02-02T21:22:44Z

If you want to recreate the problem do the following, To make every dependency shared library in appropriate hierarchy:

p.list

runtime sync
runtime/race
unicode
unicode/utf8
unicode/utf16
math/bits
math
internal/unsafeheader
internal/reflectlite
internal/testlog
container/list
container/ring
crypto/internal/subtle
crypto/subtle
vendor/golang.org/x/crypto/cryptobyte/asn1
internal/nettrace
vendor/golang.org/x/crypto/internal/subtle
encoding
internal/goversion
hash/maphash
image/color
internal/cfg
image/color/palette
math/cmplx
runtime/metrics
internal/singleflight
internal/sysinfo
math/rand
errors
sort
internal/oserror
strconv
path
vendor/golang.org/x/net/dns/dnsmessage
syscall
container/heap
hash
bytes
crypto/internal/randutil
strings
text/tabwriter
hash/adler32
hash/crc32
crypto/hmac
hash/crc64
hash/fnv
vendor/golang.org/x/crypto/hkdf
crypto
crypto/rc4
reflect
encoding/ascii85
encoding/base32
vendor/golang.org/x/text/transform
bufio
go/build/constraint
regexp/syntax
html
compress/bzip2
image
internal/syscall/execenv
plugin
internal/syscall/unix
time
time/tzdata

test.bash

#! /bin/bash
IFS=$'\n'
for i in `cat p.list`
do
go install -v -ldflags '-s -w' -gcflags '-trimpath /usr/src/go' -buildmode=shared -linkshared -pkgdir /usr/src/lib $i
done

What is expected:

/usr/src/lib/libbufio.so
/usr/src/lib/libbytes.so
/usr/src/lib/libcompress-bzip2.so
/usr/src/lib/libcontainer-heap.so
/usr/src/lib/libcontainer-list.so
/usr/src/lib/libcontainer-ring.so
/usr/src/lib/libcrypto-hmac.so
/usr/src/lib/libcrypto-internal-randutil.so
/usr/src/lib/libcrypto-internal-subtle.so
/usr/src/lib/libcrypto-rc4.so
/usr/src/lib/libcrypto.so
/usr/src/lib/libcrypto-subtle.so
/usr/src/lib/libencoding-ascii85.so
/usr/src/lib/libencoding-base32.so
/usr/src/lib/libencoding.so
/usr/src/lib/liberrors.so
/usr/src/lib/libgo-build-constraint.so
/usr/src/lib/libhash-adler32.so
/usr/src/lib/libhash-crc32.so
/usr/src/lib/libhash-crc64.so
/usr/src/lib/libhash-fnv.so
/usr/src/lib/libhash-maphash.so
/usr/src/lib/libhash.so
/usr/src/lib/libhtml.so
/usr/src/lib/libimage-color-palette.so
/usr/src/lib/libimage-color.so
/usr/src/lib/libimage.so
/usr/src/lib/libinternal-cfg.so
/usr/src/lib/libinternal-goversion.so
/usr/src/lib/libinternal-nettrace.so
/usr/src/lib/libinternal-oserror.so
/usr/src/lib/libinternal-reflectlite.so
/usr/src/lib/libinternal-singleflight.so
/usr/src/lib/libinternal-syscall-execenv.so
/usr/src/lib/libinternal-syscall-unix.so
/usr/src/lib/libinternal-sysinfo.so
/usr/src/lib/libinternal-testlog.so
/usr/src/lib/libinternal-unsafeheader.so
/usr/src/lib/libmath-bits.so
/usr/src/lib/libmath-cmplx.so
/usr/src/lib/libmath-rand.so
/usr/src/lib/libmath.so
/usr/src/lib/libpath.so
/usr/src/lib/libplugin.so
/usr/src/lib/libreflect.so
/usr/src/lib/libregexp-syntax.so
/usr/src/lib/libruntime-metrics.so
/usr/src/lib/libruntime-race.so
/usr/src/lib/libruntime,sync.so
/usr/src/lib/libsort.so
/usr/src/lib/libstrconv.so
/usr/src/lib/libstrings.so
/usr/src/lib/libsyscall.so
/usr/src/lib/libtext-tabwriter.so
/usr/src/lib/libtime.so
/usr/src/lib/libtime-tzdata.so <--
/usr/src/lib/libunicode.so
/usr/src/lib/libunicode-utf16.so
/usr/src/lib/libunicode-utf8.so
/usr/src/lib/libvendor-golang.org-x-crypto-cryptobyte-asn1.so
/usr/src/lib/libvendor-golang.org-x-crypto-hkdf.so
/usr/src/lib/libvendor-golang.org-x-crypto-internal-subtle.so
/usr/src/lib/libvendor-golang.org-x-net-dns-dnsmessage.so
/usr/src/lib/libvendor-golang.org-x-text-transform.so

What we get instead:

/tmp/go-build3257893882/b017/libtime-tzdata.so

/usr/src/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-599239160/go.o: relocation R_X86_64_PC32 against undefined symbol `time.registerLoadFromEmbeddedTZData' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

How to get around this

Building time and time/tzdata into one shared library

go install -v -ldflags '-s -w' -gcflags '-trimpath /usr/src/go' -buildmode=shared -linkshared -pkgdir /usr/src/lib time time/tzdata

Reason for doing this

Actually I understand it may be against Go language's idioms, as the point is to create static executables, or as my colleagues asked why not just a standard library as shared library libstd.so, which is of course the default approach

Yes you are all right, I am testing the new Linker in 1.16, as our code contains a ton of shared objects due to Licenses of those libraries,

I recreate the issues we faced with our developed libraries with standard libraries, when I see a problem in shared objects in our properietary code

kiap · 2021-02-03T02:39:41Z

I have the same problem with these:

internal/execabs
internal/goroot
go/build
go/internal/gccgoimporter
go/internal/srcimporter

valyala · 2021-02-14T22:43:10Z

Probably related issue, which appears when cross-compiling a Go program for GOARCH=arm on GOARCH=amd64 with timetzdata build tag:

GOARCH=arm go build -tags timetzdata ./path/to/app
time/tzdata.init.0: unresolved inter-package jump to time.registerLoadFromEmbeddedTZData(time) from time/tzdata

The program is built without issues for GOARCH={arm64,386}, while GOARCH=ppc64le emits the same error.

…ata into the app The approach with `timetzdata` build tag didn't work for GOARCH=arm and GOARCH=ppc64le due to the issue golang/go#44073 (comment)

gopherbot · 2021-02-16T15:31:37Z

Change https://golang.org/cl/292490 mentions this issue: cmd/link: generate trampoline for inter-dependent packages

cherrymui · 2021-02-16T15:36:29Z

@valyala thanks for reporting. This is a different problem. The CL above should fix it.

Currently, in the trampoline generation pass we expect packages are laid out in dependency order, so a cross-package jump always has a known target address so we can check if a trampoline is needed. With linknames, there can be cycles in the package dependency graph, making this algorithm no longer work. For them, as the target address is unkown we conservatively generate a trampoline. This may generate unnecessary trampolines (if the packages turn out laid together), but package cycles are extremely rare so this is fine. Updates #44073. Change-Id: I2dc2998edacbda27d726fc79452313a21d07787a Reviewed-on: https://go-review.googlesource.com/c/go/+/292490 Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com>

seankhliao · 2021-10-27T21:54:23Z

Obsoleted by #47788

seankhliao added the NeedsInvestigation label Feb 2, 2021

ianlancetaylor changed the title ~~pkg/tool/linux_amd64/link: Linker problem with shared time/tzdata~~ cmd/link: can't build shared time/tzdata Feb 2, 2021

cherrymui mentioned this issue Feb 26, 2021

cmd/link: fail to build when using time/tzdata on ARM #44639

Closed

dmitshur added this to the Backlog milestone Mar 1, 2021

seankhliao closed this as completed Oct 27, 2021

golang locked and limited conversation to collaborators Oct 27, 2022

gopherbot added the FrozenDueToAge label Oct 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/link: can't build shared time/tzdata #44073

cmd/link: can't build shared time/tzdata #44073

kiap commented Feb 2, 2021 •

edited by seankhliao

Loading

seankhliao commented Feb 2, 2021

cherrymui commented Feb 2, 2021

kiap commented Feb 2, 2021

seankhliao commented Feb 2, 2021

kiap commented Feb 2, 2021 •

edited

Loading

kiap commented Feb 3, 2021

valyala commented Feb 14, 2021

gopherbot commented Feb 16, 2021

cherrymui commented Feb 16, 2021

seankhliao commented Oct 27, 2021

cmd/link: can't build shared time/tzdata #44073

cmd/link: can't build shared time/tzdata #44073

Comments

kiap commented Feb 2, 2021 • edited by seankhliao Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

How to solve the issue without shared linking?

However

seankhliao commented Feb 2, 2021

cherrymui commented Feb 2, 2021

kiap commented Feb 2, 2021

seankhliao commented Feb 2, 2021

kiap commented Feb 2, 2021 • edited Loading

p.list

test.bash

What is expected:

What we get instead:

/tmp/go-build3257893882/b017/libtime-tzdata.so

How to get around this

Reason for doing this

kiap commented Feb 3, 2021

valyala commented Feb 14, 2021

gopherbot commented Feb 16, 2021

cherrymui commented Feb 16, 2021

seankhliao commented Oct 27, 2021

kiap commented Feb 2, 2021 •

edited by seankhliao

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

kiap commented Feb 2, 2021 •

edited

Loading