Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/link: error building standard lib using -buildmode=shared Go 1.20rc1 #57334

Closed
derekparker opened this issue Dec 15, 2022 · 11 comments
Closed

cmd/link: error building standard lib using -buildmode=shared Go 1.20rc1 #57334

derekparker opened this issue Dec 15, 2022 · 11 comments
Assignees
@cherrymui
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone
Go1.20

Comments

@derekparker
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.20rc1 linux/amd64

Does this issue reproduce with the latest release?

No, it does not reproduce with the latest 1.19.x release.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/deparker/.cache/go-build"
GOENV="/home/deparker/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/deparker/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/deparker/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/deparker/Code/goroot"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/deparker/Code/goroot/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20rc1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1198116115=/tmp/go-build -gno-record-gcc-switches"

What did you do?

pushd src
./make.bash
popd
GOROOT=$(pwd) ./bin/go install -buildmode=shared -v -x std

What did you expect to see?

No error building shared stdlib.

What did you see instead?

https://gist.github.com/derekparker/d391d9bb492f9ee611c8ad136e813ab3
@derekparker derekparker changed the title affected/package: cmd/link: error building standard lib using -buildmode=shared Go 1.20rc1 Dec 15, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Dec 15, 2022
@derekparker
Copy link
Contributor Author

cc @alexsaezm (found this issue while building Go 1.20rc1 for Fedora)

cc @ianlancetaylor @randall77 @heschi

@ianlancetaylor
Copy link
Member

CC @cherrymui @thanm

@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker labels Dec 15, 2022
@ianlancetaylor ianlancetaylor added this to the Go1.20 milestone Dec 15, 2022
@cherrymui
Copy link
Member

cherrymui commented Dec 15, 2022 •
edited
Loading

go install -buildmode=shared -v -x std

This command includes all packages in std, in particular, both runtime/race/internal/amd64v1 and runtime/race/internal/amd64v3. Both of these packages contain the race syso, so both syso's get into the link, causing duplicated symbols. This seem all but the last error.

In normal build, only the runtime/race package is directly imported, and one of the internal packages is actually imported based on the GOAMD64 setting. But this package selection doesn't work for go install -buildmode=shared std.

I wonder if go install -buildmode=shared std could start with all user-importable packages in std and only pull in internal packages on demand...

/usr/bin/ld: /tmp/go-link-2738326382/go.o: relocation R_X86_64_PC32 against undefined symbol `runtime_test.callerStartLine' can not be used when making a shared object; recompile with -fPIC

The last error is due to a package that is used only in test, and calls a function defined in test. Again building std includes this package, but not the function defined in test. User cannot import that package. We could probably work around this one. Pulling only user-importable part of std would help this as well.

@ianlancetaylor
Copy link
Member

CC @bcmills

@bcmills
Copy link
Contributor

bcmills commented Dec 16, 2022

For the runtime/race/internal packages, would it suffice to add go:build constraints to the doc.go files so that only one or the other package actually exists? (Or does the package exist because of the syso files even if all of the .go files are constrained away?)

@bcmills
Copy link
Contributor

bcmills commented Dec 16, 2022

For the test-only package, does it suffice to move that package inside a testdata directory? It will probably still be importable...

@alexsaezm
Copy link
Contributor

For the record, there are also "similar" behaviors in other architectures:

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/458696 mentions this issue: runtime/internal/startlinetest: work around shared buildmode linking issue

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/458695 mentions this issue: runtime/race: add build tag to internal amd64vN packages

@cherrymui
Copy link
Member

would it suffice to add go:build constraints to the doc.go files so that only one or the other package actually exists?

Thanks. This works.

For the other issue I worked around it using a function pointer.

@mknyszek mknyszek moved this to In Progress in Go Compiler / Runtime Dec 21, 2022
gopherbot pushed a commit that referenced this issue Dec 22, 2022
@cherrymui
runtime/race: add build tag to internal amd64vN packages

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
18baca6 
Only one of the runtime/race/internal/amd64vN packages should be
included in a build. Generally this is true because the
runtime/race package would import only one of them depending on
the build configuration. But for "go install -buildmode=shared std"
it includes all Go packages in std, which includes both, which
then causes link-time failure due to duplicated symbols. To avoid
this, we add build tags to the internal packages, so, depending on
the build configuation, only one package would contain buildable
go files therefore be included in the build.

For #57334.

Change-Id: I52ddc3a40e16c7d04b4dd861e9689918d27e8509
Reviewed-on: https://go-review.googlesource.com/c/go/+/458695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Repository owner moved this from In Progress to Done in Go Compiler / Runtime Dec 22, 2022
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/459056 mentions this issue: misc/cgo/testshared: test build std in shared mode

gopherbot pushed a commit that referenced this issue Dec 22, 2022
@cherrymui
misc/cgo/testshared: test build std in shared mode
6d3139b 
Test that "go install -buildmode=shared std" works.

For #57334.

Change-Id: I465a07cf2e9035995916ef9940b4c1eeba998099
Reviewed-on: https://go-review.googlesource.com/c/go/+/459056
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tim Scharfenort <timscharfenort89@googlemail.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
@golang golang locked and limited conversation to collaborators Dec 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@cherrymui cherrymui

Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Milestone
Go1.20
Development

No branches or pull requests
6 participants
@derekparker @alexsaezm @ianlancetaylor @bcmills @gopherbot @cherrymui