Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: locating GOROOT fails when the go command is run from the cross-compiled bin subdirectory #62119

Closed
tspearconquest opened this issue Aug 17, 2023 · 12 comments
Closed

cmd/go: locating GOROOT fails when the go command is run from the cross-compiled bin subdirectory #62119

tspearconquest opened this issue Aug 17, 2023 · 12 comments
Assignees
@bcmills
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.22

Comments

@tspearconquest
Copy link

tspearconquest commented Aug 17, 2023 •
edited

What version of Go are you using (go version)?

$ /usr/local/go/bin/go version
go version go1.21.0 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ /usr/local/go/bin/go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/go/.cache/go-build'
GOENV='/go/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='[https://proxy.golang.org,direct'](https://proxy.golang.org%2Cdirect%27/)
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2492856274=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I'm trying to build an internal golang image based originally on the docker-library dockerfile for golang 1.20, but customized in the following ways:

  1. Retrieve the sources for 1.21 instead of 1.20
  2. Build a multi-architecture (AMD64 and ARM64) image as a single image with Buildkit
  3. Use a multi-stage build process to copy only the relevant files from the build stage to the final stage; the final stage runs only busybox and has no package manager
  4. Do the whole build without root privileges except for the package installs and adding a non-root user to the base image
  5. Do a simple smoke test of the go binary before publishing the image to our internal registry (this is where the build is failing now)
  6. Do some basic setup of GOPATH to set strict permissions on it while allowing the non-root user to write where it needs to for our developers' pipelines

The full dockerfile with some redaction of internal repo paths:

# syntax=docker/dockerfile:1

FROM --platform=${BUILDPLATFORM} registry.gitlab.com/my-corp/images/alpine-image/alpine:stable as builder
USER 0

ARG IMAGE_VERSION
ENV GOLANG_VERSION=${IMAGE_VERSION}
ENV PATH=/usr/local/go/bin:${PATH}

ARG BUILDARCH
RUN --mount=type=cache,id=apk-${BUILDARCH},sharing=locked,target=/var/cache/apk \
    ln -vsf /var/cache/apk /etc/apk/cache \
 && apk --update add \
    bash==5.2.15-r5 \
    gcc==12.2.1_git20220924-r10 \
    git==2.40.1-r0 \
    go==1.20.7-r0 \
    musl-dev==1.2.4-r1

USER 65532:65532
ARG TARGETARCH
WORKDIR /usr/local/go
RUN git clone -b "go${GOLANG_VERSION}" --depth 1 https://github.com/golang/go.git . \
 && export GOCACHE='/tmp/gocache' \
 && ( \
      cd /usr/local/go/src \
   && export  \
      GOAMD64='v1' \
      GOARCH="${TARGETARCH}" \
      GOOS='linux' \
      GOROOT_BOOTSTRAP="$(go env GOROOT)" \
      GOHOSTOS='linux' \
      GOHOSTARCH="${BUILDARCH}" \
   && ./make.bash \
    ) \
 && rm -rf \
    /usr/local/go/pkg/*/cmd \
    /usr/local/go/pkg/bootstrap \
    /usr/local/go/pkg/obj \
    /usr/local/go/pkg/tool/*/api \
    /usr/local/go/pkg/tool/*/go_bootstrap \
    /usr/local/go/src/cmd/dist/dist \
    "${GOCACHE}"

USER 0
RUN addgroup -S -g 65532 golang \
 && adduser -S -D -H -h /go -u 65532 -G golang golang

USER 65532:65532

FROM registry.gitlab.com/my-corp/images/busybox-image/busybox:stable as image
USER 0

ARG IMAGE_VERSION
ENV GOCACHE=/go/.cache/go-build
ENV GOENV=/go/.config/go/env
ENV GOLANG_VERSION=${IMAGE_VERSION}
ENV GOPATH=/go
# Added this and things work, so just want to confirm if this is the correct fix
# ENV GOROOT=/go
ARG TARGETARCH
ENV PATH=${GOPATH}/bin:/usr/local/go/bin/linux_${TARGETARCH}:/usr/local/go/bin:${PATH}

COPY --link --from=builder /etc/passwd /etc/passwd
COPY --link --from=builder /etc/group /etc/group
COPY --link --from=builder --chown=0:0 /usr/local/go /usr/local/go
RUN go version \ ## failing here
 && mkdir ${GOPATH} \
 && mkdir -p ${GOPATH}/.cache ${GOPATH}/.config/go ${GOPATH}/bin ${GOPATH}/pkg/mod ${GOPATH}/pkg/sumdb ${GOPATH}/src \
 && touch ${GOPATH}/.netrc \
 && chmod g+w ${GOPATH}/.netrc \
 && chmod 0755 ${GOPATH} \
 && chmod 1775 ${GOPATH}/.cache ${GOPATH}/.config/go ${GOPATH}/bin ${GOPATH}/pkg/mod ${GOPATH}/pkg/sumdb ${GOPATH}/src \
 && chgrp golang ${GOPATH}/.cache ${GOPATH}/.config/go ${GOPATH}/.netrc ${GOPATH}/bin ${GOPATH}/pkg/mod ${GOPATH}/pkg/sumdb ${GOPATH}/src

WORKDIR ${GOPATH}

USER 65532:65532

ENTRYPOINT [ "go" ]
CMD [ "version" ]

What did you expect to see?

go version output the version

What did you see instead?

The same error as in #61921
@dmitshur
Copy link
Contributor

CC @bcmills, @golang/release.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 17, 2023
@dmitshur dmitshur added this to the Backlog milestone Aug 17, 2023
@tspearconquest tspearconquest mentioned this issue Aug 17, 2023
Closed
@bcmills bcmills changed the title cmd/dist: go: cannot find GOROOT directory: 'go' binary is trimmed and GOROOT is not set cmd/go: locating GOROOT fails when the go command is run from the cross-compiled bin subdirectory Aug 17, 2023
@bcmills bcmills self-assigned this Aug 17, 2023
@bcmills bcmills added the GoCommand cmd/go label Aug 17, 2023
@bcmills
Copy link
Contributor

bcmills commented Aug 17, 2023

The cross-compiling configuration seems to be the key here. I can reproduce the failure mode by running make.bash with GOARCH=386 GOHOSTARCH=amd64 and then running the cross-compiled go command explicitly:

~/go/src$ go install -trimpath cmd

~/go/src$ ../bin/linux_386/go env GOROOT
go: cannot find GOROOT directory: 'go' binary is trimmed and GOROOT is not set

@tspearconquest
Copy link
Author

Is this a legitimate issue or something I need to fix on my end; and if it's the latter, what's the proper way to handle a cross compile?

@bcmills
Copy link
Contributor

bcmills commented Aug 17, 2023

It's a bug in cmd/go, I think, but kind of an obscure one. I don't really understand how multi-architecture images work, so it's hard to say beyond that. 😅

@tspearconquest
Copy link
Author

Thanks! If we can help troubleshoot in any way, please let me know. Our infrastructure and my mac both run Intel/AMD but one of my colleagues (cc: @waterfoul) has an M2 mac, so we have the ability to test on either architecture without affecting anything.

@gopherbot
Copy link

Change https://go.dev/cl/521015 mentions this issue: cmd/go: find GOROOT using os.Executable when installed to GOROOT/bin/GOOS_GOARCH

@bcmills
Copy link
Contributor

bcmills commented Aug 18, 2023

@gopherbot, please backport to Go 1.21. This interacts badly with the change to build cmd/go with -trimpath by default, causing cross-compiled go commands to fail to locate GOROOT if built at a release version.

@gopherbot
Copy link

Backport issue(s) opened: #62144 (for 1.21).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@gopherbot gopherbot mentioned this issue Aug 18, 2023
Closed
@bcmills
Copy link
Contributor

bcmills commented Aug 18, 2023 •
edited

@tspearconquest, this should be fixed by https://go.dev/cl/521015 when that is merged.

In the meantime, you could potentially work around it by deleting the $GOROOT/VERSION file before building the toolchain, which will cause GOROOT to once again be embedded in the binaries.

@bcmills bcmills modified the milestones: Backlog, Go1.22 Aug 18, 2023
@ardnew
Copy link

ardnew commented Aug 18, 2023

Just got bit by this using a GOOS=windows GOARCH=amd64 cross-compiled build (hosted on WSL) with some commit on the go1.21 branch. The error appeared with VS Code failing to update gopls by claiming to not have GOROOT defined:

# in WSL
cd $GOROOT; GOOS=windows GOARCH=amd64 ./src/make.bash

Then VS Code on Windows:

Installing 1 tool at C:\andrew\go\bin in module mode.
  gopls

Installing golang.org/x/tools/gopls@latest FAILED
{
 "code": 2,
 "killed": false,
 "signal": null,
 "cmd": "C:\\andrew\\go\\dev\\bin\\windows_amd64\\go.exe install -v golang.org/x/tools/gopls@latest",
 "stdout": "",
 "stderr": "go: cannot find GOROOT directory: 'go' binary is trimmed and GOROOT is not set\n"
}

1 tools failed to install.

gopls: failed to install gopls(golang.org/x/tools/gopls@latest): Error: Command failed: C:\andrew\go\dev\bin\windows_amd64\go.exe install -v golang.org/x/tools/gopls@latest
go: cannot find GOROOT directory: 'go' binary is trimmed and GOROOT is not set

I've deleted $GOROOT/VERSION and rebuilt, but not seeing any difference in behavior; receiving the same error messages.

@gopherbot
Copy link

Change https://go.dev/cl/521695 mentions this issue: [release-branch.go1.21] cmd/go: find GOROOT using os.Executable when installed to GOROOT/bin/GOOS_GOARCH

cellularmitosis pushed a commit to cellularmitosis/go that referenced this issue Aug 24, 2023
@cellularmitosis
cmd/go: find GOROOT using os.Executable when installed to GOROOT/bin/…
7f6aecc 
…GOOS_GOARCH

When running make.bash in a cross-compiled configuration
(for example, GOARCH different from GOHOSTARCH), cmd/go
is installed to GOROOT/bin/GOOS_GOARCH instead of GOROOT/bin.

That means that we need to look for GOROOT in both ../.. and ../../..,
not just the former.

Fixes golang#62119.
Updates golang#18678.

Change-Id: I283c6a10c46df573ff44da826f870417359226a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/521015
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit that referenced this issue Aug 30, 2023
@gopherbot
[release-branch.go1.21] cmd/go: find GOROOT using os.Executable when …
6385a6f 
…installed to GOROOT/bin/GOOS_GOARCH

When running make.bash in a cross-compiled configuration
(for example, GOARCH different from GOHOSTARCH), cmd/go
is installed to GOROOT/bin/GOOS_GOARCH instead of GOROOT/bin.

That means that we need to look for GOROOT in both ../.. and ../../..,
not just the former.

Fixes #62144.
Updates #62119.
Updates #18678.

Change-Id: I283c6a10c46df573ff44da826f870417359226a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/521015
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 9e9556d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/521695
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
@Torrekie

This comment was marked as resolved.

Sign in to view
@bcmills bcmills mentioned this issue Jan 4, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bcmills bcmills

Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.22
Development

No branches or pull requests
6 participants
@dmitshur @ardnew @bcmills @gopherbot @Torrekie @tspearconquest