Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/nm: TestCgoLib fails if CGO_CFLAGS contains unrecognized flags (tool nm can't process .a file) #62036

Closed
fastcat opened this issue Aug 15, 2023 · 3 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.

Comments

@fastcat
Copy link

fastcat commented Aug 15, 2023

What version of Go are you using (go version)?

$ go version
1.21.0

Does this issue reproduce with the latest release?

1.21.0 is the latest release. The problematic code is still present on master. This also reproduces in go 1.20, and probably in go 1.19 (based on the backport status of the CL below, though I have not directly confirmed that)

What operating system and processor architecture are you using (go env)?

Linux amd64

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cheetah/.cache/go-build"
GOENV="/home/cheetah/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/data/cheetah/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/data/cheetah/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.19"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.19/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.8"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build505240055=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Trying to build go from Debian source package. The Debian packaging sometimes, as part of their reproducible builds effort, causes some extra CGO_CFLAGS (and CGO_CXXFLAGS) to be set in the environment in addition to what is above. Notably this includes -ffile-prefix-map=/some/source/dir=..

Note that while Debian-specific details are the particular way I discovered this, the underlying issue -- that the go linker can generate .a files that go tool nm cannot read -- is not specific to that scenario.

The presence of this flag causes the .a generation for the TestCgoLib test, building a library with -buildmode=archive, to include a preferlinkext empty sigil file (see #58619 and https://go.dev/cl/475375). However go tool nm is unable to process this sigil within the .a file, and so it errors out when trying to read the symbols during the test.

This happens because the .a entry listing ends up in this switch case:

case archive.EntryNativeObj:
nr := io.NewSectionReader(f, e.Offset, e.Size)
for _, try := range openers {
if raw, err := try(nr); err == nil {
entries = append(entries, &Entry{
name: e.Name,
raw: raw,
})
continue L
}
}

Since preferlinkext is an empty file, of course no binary opener (ELF or otherwise) is going to succeed at reading any entries from it, and so it falls through to:
return nil, fmt.Errorf("open %s: unrecognized archive member %s", f.Name(), e.Name)

After that it tries to process the .a file with the ELF and other "openers", which of course all fail, and so it finally fails with unrecognized object file at:

return nil, fmt.Errorf("open %s: unrecognized object file", name)

What did you expect to see?

go tool nm and its tests should work even if the preferlinkext flag ends up in a .a file

What did you see instead?

go tool nm and its tests fail if the preferlinkext flag ends up in a .a file, causing the tests to fail:

--- FAIL: TestCgoLib (2.19s)
    nm_test.go:264: go tool nm: exit status 1
        open /tmp/TestGoLib2084668406/gopath/src/mylib/mylib.a: unrecognized object file

I think it should be easy enough to recognize the flag file in the switch case above and just skip it?

cc @thanm who wrote the CL above that introduced this flag file


A Dockerfile that can be used to reproduce the issue in isolation:

FROM debian:bookworm

RUN apt-get update
RUN apt-get -y --no-install-recommends install build-essential
RUN apt-get -y --no-install-recommends install git
RUN apt-get -y --no-install-recommends install git-buildpackage
RUN apt-get -y --no-install-recommends install debhelper-compat=13 golang-go netbase

# commenting out the next line line will cause the problem CGO_CFLAGS to not be set and thus the build will pass
RUN apt-get -y --no-install-recommends install dh-golang

RUN git clone --branch=golang-1.21 --filter=blob:none https://salsa.debian.org/go-team/compiler/golang.git
WORKDIR /golang
RUN gbp buildpackage -b -uc -us -rfakeroot --git-ignore-new || true
RUN cp -v /tmp/nm.test /tmp/mylib.a /
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Aug 15, 2023
@fastcat
Copy link
Author

fastcat commented Aug 15, 2023

https://go.dev/cl/504335 for #60865 looks like it would/will mask this issue by permitting the specific flag causing trouble in this scenario (-ffile-prefix-map) to not trigger external linking, but the underlying go tool nm issue would remain, just be harder to trip over

@thanm thanm self-assigned this Aug 15, 2023
@thanm
Copy link
Contributor

thanm commented Aug 15, 2023

Thanks for the report. I'll look into this.

@gopherbot
Copy link

Change https://go.dev/cl/519635 mentions this issue: cmd/internal/archive: skip sentinel archive entries created by Go cmd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
None yet
Development

No branches or pull requests

3 participants