cmd/compile: crypto/elliptic build error under -linkshared mode #58966

quaresmajose · 2023-03-10T15:47:38Z

What version of Go are you using (`go version`)?

$ go version
go version go1.20.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

This issue can be reproduced inside the official golang:1.20-alpine docker container and it works fine on the previous golang:1.19-alpine

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/build/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3925415831=/tmp/go-build -gno-record-gcc-switches"

What did you do?

This error has been discovered in Yocto/OpenEmbedded that builds the go shared std library and this issue also affect
other go projects [1], the workaround to build successfully since go 1.20 is not using the -linkshared

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310

Steps to reproduce:

# fetch some affected source
git clone https://github.com/foundriesio/fioconfig && cd fioconfig

# go-1.19 is good
cat > Dockerfile.shared << EOF
FROM golang:1.19-alpine
RUN mkdir /build
WORKDIR /build
RUN apk add --update --no-cache g++
COPY . .
RUN go mod download
RUN go install -buildmode=shared -linkshared std
RUN go build -linkshared -tags vpn -o ./fioconfig main.go
EOF
docker build . -f Dockerfile.shared

# go-1.20 is broken
cat > Dockerfile.shared << EOF
FROM golang:1.20-alpine
RUN mkdir /build
WORKDIR /build
RUN apk add --update --no-cache g++
COPY . .
RUN go env
RUN go mod download
RUN go install -buildmode=shared -linkshared std
RUN go build -linkshared -tags vpn -o ./fioconfig main.go
EOF
docker build . -f Dockerfile.shared

What did you expect to see?

Build my go linkshared application without any errors like on previous version 1.19.

What did you see instead?

Linking errors

# command-line-arguments
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Add
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Add
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).CombinedMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).CombinedMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Double
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Double
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).IsOnCurve
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).IsOnCurve
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Params
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Params
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarBaseMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarBaseMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Unmarshal
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Unmarshal
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).UnmarshalCompressed
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).UnmarshalCompressed
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).normalizeScalar
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).normalizeScalar
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).pointFromAffine
/usr/local/go/pkg/tool/linux_amd64/link: too many errors
The command '/bin/sh -c go build -linkshared -tags vpn -o ./fioconfig main.go' returned a non-zero code: 1

The text was updated successfully, but these errors were encountered:

quaresmajose · 2023-03-10T15:58:50Z

It looks like the crypto/elliptic is now no more than a compatibility wrapper and the new crypto/ecdh don't implement the P-224 curves [2].

[2] https://words.filippo.io/dispatches/go-1-20-cryptography

@FiloSottile is the author of the blog post so he maybe know better what going on.

cagedmantis · 2023-03-10T16:41:04Z

cc @rolandshoemaker @golang/security

mdempsky · 2023-03-14T06:45:10Z

Standalone repro:

$ cat x.go
package main

import "crypto/elliptic"

var curve elliptic.Curve

func main() {
	switch curve {
	case elliptic.P224():
	}
}
$ go install -linkshared -buildmode=shared std
$ go build -linkshared x.go
# command-line-arguments
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Add
...

I think this is a compiler or linker issue, not an issue with crypto/elliptic itself.

I expect the issue has to do with instantiating generic types across DSO boundaries (due to inlining).

/cc @golang/compiler

mdempsky · 2023-03-14T21:37:08Z

@cherrymui gave this explanation:

Okay, I think I understand it a little bit. We just don't support a type symbol referencing a method symbol across DSO boundary. That reference, R_METHODOFF, is relative to the section start. But we cannot do it across the DSO boundary, because we can't reference the section in a different DSO. Also, the runtime code resolves the section-relative offset to the same runtime module that contains the type symbol.

So, either the type symbol and the method symbol are both in the shared object, or both in the main executable. But not cross.

cuonglm · 2023-03-15T01:00:52Z

FYI, this is started failing since https://go-review.googlesource.com/c/go/+/443535 (with 18baca6 and de6abd7 applied for fixing go install -linkshared -buildmode=shared std)

quaresmajose · 2023-03-15T09:36:15Z

The described steps to reproduce this issue uses GOARCH="amd64" but please note that it can be reproduced on other architectures too. At last I can reproduce the issue on arm and arm64 so maybe this is architecture independent.

gopherbot · 2023-03-15T21:50:00Z

Change https://go.dev/cl/476735 mentions this issue: cmd/link: improve error message and debugging

cuonglm · 2023-03-17T03:23:21Z

@cherrymui gave this explanation:

Okay, I think I understand it a little bit. We just don't support a type symbol referencing a method symbol across DSO boundary. That reference, R_METHODOFF, is relative to the section start. But we cannot do it across the DSO boundary, because we can't reference the section in a different DSO. Also, the runtime code resolves the section-relative offset to the same runtime module that contains the type symbol.
So, either the type symbol and the method symbol are both in the shared object, or both in the main executable. But not cross.

@mdempsky maybe when building with -linkshared, we should always apply InlineCalls even the instantiated function was compiled in imported package?

gopherbot · 2023-03-17T18:02:38Z

Change https://go.dev/cl/477375 mentions this issue: cmd/compile: re-compile instantiated generic functions in linkshared mode

Correct an error message to missing section, not unreachable symbol. Also, under -v >= 2, dump symbol info on error for debugging. Updates #58966. Change-Id: I0f832c517d64f4b672b313a8b9be2d028744f945 Reviewed-on: https://go-review.googlesource.com/c/go/+/476735 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>

Since go 1.20 some packages fails to link [1][2] and only build staticaly with GO_LINKSHARED = "" [1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310 [2] golang/go#58966 Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

Since go 1.20 some packages fails to link [1][2] and only build staticaly with GO_LINKSHARED = "" [1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310 [2] golang/go#58966 (From OE-Core rev: afb3fc1701988dfcd79191343b49f4db0966236a) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

Since go 1.20 some packages fails to link [1][2] and only build staticaly with GO_LINKSHARED = "" [1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310 [2] golang/go#58966 (From OE-Core rev: d6a010a9d62fc1ae75935c0d0664b8353eede756) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

Since go 1.20 some packages fails to link [1][2] and only build staticaly with GO_LINKSHARED = "" [1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310 [2] golang/go#58966 (From OE-Core rev: eb40d6c2ef72814336365208722e2dc96322e07d) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Since go 1.20 some packages fails to link [1][2] and only build staticaly with GO_LINKSHARED = "" [1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310 [2] golang/go#58966 Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

cuonglm · 2023-03-25T05:22:22Z

@gopherbot please backport this issue to go1.20

gopherbot · 2023-03-25T05:22:48Z

Backport issue(s) opened: #59236 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

This reverts commit d2a630c. The linkshared is fixed upstream [1] and backported in oe-core [2] [1] golang/go#58966 [2] https://git.yoctoproject.org/poky/commit/?id=d5c79ca6a77af1a04992fff4300333e02d94d84d Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

The linkshared is fixed upstream [1] and backported in oe-core [2] [1] golang/go#58966 [2] https://git.yoctoproject.org/poky/commit/?id=d5c79ca6a77af1a04992fff4300333e02d94d84d Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

cagedmantis added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 10, 2023

cagedmantis added this to the Backlog milestone Mar 10, 2023

mdempsky changed the title ~~crypto/elliptic: linkshared regression in crypto/internal/nistec.P224Point since version 1.20~~ cmd/compile: crypto/elliptic build error under -linkshared mode Mar 14, 2023

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Mar 14, 2023

mknyszek assigned cherrymui Mar 15, 2023

cherrymui assigned mdempsky Mar 15, 2023

gopherbot closed this as completed in bcd8212 Mar 22, 2023

gopherbot mentioned this issue Mar 25, 2023

cmd/compile: crypto/elliptic build error under -linkshared mode [1.20 backport] #59236

Closed

dmitshur modified the milestones: Backlog, Go1.21 Mar 29, 2023

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 29, 2023

vkhoroz mentioned this issue Sep 26, 2023

(chore): Update go version from 1.19 to 1.21 and dockerfile foundriesio/fioctl#271

Closed

golang locked and limited conversation to collaborators Mar 28, 2024

gopherbot added the FrozenDueToAge label Mar 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/compile: crypto/elliptic build error under -linkshared mode #58966

cmd/compile: crypto/elliptic build error under -linkshared mode #58966

quaresmajose commented Mar 10, 2023 •

edited

quaresmajose commented Mar 10, 2023 •

edited

cagedmantis commented Mar 10, 2023

mdempsky commented Mar 14, 2023 •

edited

mdempsky commented Mar 14, 2023

cuonglm commented Mar 15, 2023

quaresmajose commented Mar 15, 2023

gopherbot commented Mar 15, 2023

cuonglm commented Mar 17, 2023

gopherbot commented Mar 17, 2023

cuonglm commented Mar 25, 2023

gopherbot commented Mar 25, 2023

cmd/compile: crypto/elliptic build error under -linkshared mode #58966

cmd/compile: crypto/elliptic build error under -linkshared mode #58966

Comments

quaresmajose commented Mar 10, 2023 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

quaresmajose commented Mar 10, 2023 • edited

cagedmantis commented Mar 10, 2023

mdempsky commented Mar 14, 2023 • edited

mdempsky commented Mar 14, 2023

cuonglm commented Mar 15, 2023

quaresmajose commented Mar 15, 2023

gopherbot commented Mar 15, 2023

cuonglm commented Mar 17, 2023

gopherbot commented Mar 17, 2023

cuonglm commented Mar 25, 2023

gopherbot commented Mar 25, 2023

quaresmajose commented Mar 10, 2023 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

quaresmajose commented Mar 10, 2023 •

edited

mdempsky commented Mar 14, 2023 •

edited