Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go mod download breaks on 1.21.0 due to empty GOPROXY #61928

Closed
Snaipe opened this issue Aug 10, 2023 · 10 comments
Closed

cmd/go: go mod download breaks on 1.21.0 due to empty GOPROXY #61928

Snaipe opened this issue Aug 10, 2023 · 10 comments
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.22

Comments

@Snaipe
Copy link

Snaipe commented Aug 10, 2023 •
edited

What version of Go are you using (go version)?

$ go version
1.21.0

Does this issue reproduce with the latest release?

Yes, and it only happens with the latest 1.21.0. Things were working by default on the previous release.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/snaipe/.local/var/cache/go-build'
GOENV='/home/snaipe/.local/etc/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/snaipe/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/snaipe/go'
GOPRIVATE=''
GOPROXY=''
GOROOT='/tmp/go'
GOSUMDB=''
GOTMPDIR=''
GOTOOLCHAIN=''
GOTOOLDIR='/tmp/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1578046652=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Run go mod download -x github.com/xdg/scram@v1.0.5

What did you expect to see?

It should have worked, and output something like this:

# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.info
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.info: 200 OK (0.086s)
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.mod
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.mod: 200 OK (0.005s)
# get https://proxy.golang.org/sumdb/sum.golang.org/supported
# get https://proxy.golang.org/sumdb/sum.golang.org/supported: 404 Not Found (0.005s)
# get https://sum.golang.org/lookup/github.com/xdg/scram@v1.0.5
# get https://sum.golang.org/lookup/github.com/xdg/scram@v1.0.5: 200 OK (0.281s)
# get https://sum.golang.org/tile/8/1/288
# get https://sum.golang.org/tile/8/1/288: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/0/x073/886
# get https://sum.golang.org/tile/8/2/001.p/33
# get https://sum.golang.org/tile/8/0/x074/141.p/165
# get https://sum.golang.org/tile/8/1/289.p/157
# get https://sum.golang.org/tile/8/2/001.p/33: 200 OK (0.010s)
# get https://sum.golang.org/tile/8/1/289.p/157: 200 OK (0.010s)
# get https://sum.golang.org/tile/8/0/x073/886: 200 OK (0.035s)
# get https://sum.golang.org/tile/8/0/x074/141.p/165: 200 OK (0.039s)
# get https://sum.golang.org/tile/8/0/x034/273
# get https://sum.golang.org/tile/8/0/x034/273: 200 OK (0.014s)
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.zip
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.zip: 200 OK (0.008s)

What did you see instead?

The command fails with this error:

go: github.com/xdg/scram@v1.0.5: GOPROXY list is not the empty string, but contains no entries

A simple way to reproduce the issue:

$ git clone https://github.com/golang/go --single-branch -b go1.21.0 --depth=1 /tmp/go/src
$ export GOROOT=/tmp/go/src
$ export GOROOT_BOOTSTRAP=/usr/lib/go
$ export GOROOT_FINAL=/tmp/go/out
$ export CGO_ENABLED=0 
$ cd /tmp/go/src/src
$ bash ./make.bash
$ cd /tmp/go
$ cp -rf src out
$ cd out
$ ./bin/go version
go version go1.21.0 linux/amd64
$ ./bin/go env GOPROXY
$ ./bin/go mod download -x github.com/xdg/scram@v1.0.5
go: github.com/xdg/scram@v1.0.5: GOPROXY list is not the empty string, but contains no entries

For comparison, using my locally installed Go 1.20.1:

$ go version
go version go1.20.1 linux/amd64
$ go env GOPROXY
https://proxy.golang.org,direct
$ go mod download -x github.com/xdg/scram@v1.0.5
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.info
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.info: 200 OK (0.086s)
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.mod
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.mod: 200 OK (0.005s)
# get https://proxy.golang.org/sumdb/sum.golang.org/supported
# get https://proxy.golang.org/sumdb/sum.golang.org/supported: 404 Not Found (0.005s)
# get https://sum.golang.org/lookup/github.com/xdg/scram@v1.0.5
# get https://sum.golang.org/lookup/github.com/xdg/scram@v1.0.5: 200 OK (0.281s)
# get https://sum.golang.org/tile/8/1/288
# get https://sum.golang.org/tile/8/1/288: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/0/x073/886
# get https://sum.golang.org/tile/8/2/001.p/33
# get https://sum.golang.org/tile/8/0/x074/141.p/165
# get https://sum.golang.org/tile/8/1/289.p/157
# get https://sum.golang.org/tile/8/2/001.p/33: 200 OK (0.010s)
# get https://sum.golang.org/tile/8/1/289.p/157: 200 OK (0.010s)
# get https://sum.golang.org/tile/8/0/x073/886: 200 OK (0.035s)
# get https://sum.golang.org/tile/8/0/x074/141.p/165: 200 OK (0.039s)
# get https://sum.golang.org/tile/8/0/x034/273
# get https://sum.golang.org/tile/8/0/x034/273: 200 OK (0.014s)
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.zip
# get https://proxy.golang.org/github.com/xdg/scram/@v/v1.0.5.zip: 200 OK (0.008s)
@bcmills
Copy link
Contributor

bcmills commented Aug 10, 2023

The default GOPROXY configuration is now read from GOROOT/go.env.

What do go env GOROOT and cat $(go env GOROOT)/go.env show when you run those commands?

@bcmills bcmills added GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Aug 10, 2023
@bcmills bcmills added this to the Go1.22 milestone Aug 10, 2023
@bcmills
Copy link
Contributor

bcmills commented Aug 10, 2023

And, on a related note: what are the contents of /home/snaipe/.local/etc/go/env?

@Snaipe
Copy link
Author

Snaipe commented Aug 10, 2023

In the freshly built Go root dir, go.env contains the following:

# This file contains the initial defaults for go command configuration.
# Values set by 'go env -w' and written to the user's go/env file override these.
# The environment overrides everything else.

# Use the Go module mirror and checksum database by default.
# See https://proxy.golang.org for details.
GOPROXY=https://proxy.golang.org,direct
GOSUMDB=sum.golang.org

# Automatically download newer toolchains as directed by go.mod files.
# See https://go.dev/doc/toolchain for details.
GOTOOLCHAIN=auto

So these values look expected. My /home/snaipe/.local/etc/go/env contains:

GOFLAGS=

I don't quite remember why I set this before, so I removed the file.

The behavior stays the same on different environments. If I build Go from source from a minimal Alpine Linux container, which should avoid any weird user config, the error remains.

@Snaipe
Copy link
Author

Snaipe commented Aug 10, 2023

I edited the initial reproducer; I realized upon rereading that I failed to transcribe it correctly.

@Snaipe
Copy link
Author

Snaipe commented Aug 10, 2023

I straced the newly built Go process and the path it tries to open go.env at seems very wrong:

LC_ALL=C strace -fe open,openat ./bin/go mod download github.com/xdg/scram@v1.0.5
openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY) = 3
strace: Process 3928185 attached
strace: Process 3928186 attached
strace: Process 3928187 attached
[pid 3928184] --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=3928184, si_uid=1000} ---
strace: Process 3928188 attached
strace: Process 3928189 attached
[pid 3928184] openat(AT_FDCWD, "/home/snaipe/.local/etc/go/env", O_RDONLY|O_CLOEXEC) = 3
[pid 3928184] openat(AT_FDCWD, "/tmp/go/go.env", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
go: github.com/xdg/scram@v1.0.5: GOPROXY list is not the empty string, but contains no entries
[pid 3928189] +++ exited with 1 +++
[pid 3928188] +++ exited with 1 +++
[pid 3928187] +++ exited with 1 +++
[pid 3928186] +++ exited with 1 +++
[pid 3928185] +++ exited with 1 +++
+++ exited with 1 +++

And sure enough, as you suspected, GOROOT seems wrong:

$ ./bin/go env GOROOT
/tmp/go

(it should have been /tmp/go/out)

Is GOROOT_FINAL no longer respected for building Go?

@Snaipe
Copy link
Author

Snaipe commented Aug 10, 2023

Nevermind, the problem was our build toolchain; I think GOROOT was incorrectly still set to the source directory. Let me investigate a bit to make sure, but it's likely not a problem with Go.

@bcmills
Copy link
Contributor

bcmills commented Aug 10, 2023

Is GOROOT_FINAL no longer respected for building Go?

Ah, see #61921.

@Snaipe
Copy link
Author

Snaipe commented Aug 10, 2023

After more investigation, the "reproducer" I had wasn't accurately representing what was happening on our end. It turns out our build instructions had a bit of a footgun that caused the go.env file to not get copied. I can confirm that things are now working as expected.

Thanks for the pointers about go.env, I don't think I'd have been able to figure what was wrong that quickly otherwise.

@Snaipe Snaipe closed this as completed Aug 10, 2023
@deadprogram deadprogram mentioned this issue Aug 12, 2023
Closed
@AeliusSaionji AeliusSaionji mentioned this issue Aug 14, 2023
Closed
7 tasks
@VioletHynes VioletHynes mentioned this issue Aug 14, 2023
Closed
@jpeeler jpeeler mentioned this issue Aug 14, 2023
Merged
9 tasks
@kevinmichaelchen kevinmichaelchen mentioned this issue Aug 14, 2023
Merged
@adam-azarchs adam-azarchs mentioned this issue Aug 17, 2023
Closed
starius added a commit to starius/gohere that referenced this issue Aug 22, 2023
@starius
include patched go.env into goroot
4710461 
Fix the following error:
go: GOPROXY list is not the empty string, but contains no entries

It started appearing after switching to Go 1.21.0.
See golang/go#61928

I changed go.env to disable settings which I consider unsafe.
starius added a commit to starius/config that referenced this issue Aug 22, 2023
@starius
gohere: fix go.env issue
9eb6803 
include patched go.env into goroot

Fix the following error:
go: GOPROXY list is not the empty string, but contains no entries

It started appearing after switching to Go 1.21.0.
See golang/go#61928

I changed go.env to disable settings which I consider unsafe.
@tcnghia tcnghia mentioned this issue Sep 21, 2023
Merged
This was referenced Jan 11, 2024
Closed
Closed
@mfrw mfrw mentioned this issue Feb 22, 2024
Merged
12 tasks
mfrw added a commit to microsoft/azurelinux that referenced this issue Feb 23, 2024
@mfrw
golang: include go.env in GOROOT (#8021)
eb81d4d 
Reference: golang/go#61928
Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
@ghasemloo
Copy link

ghasemloo commented Apr 14, 2024 •
edited

if you ran into this issue, try unsetting your GOROOT:

unset GOROOT

It fixed the issue for me on my old laptop where I hit this issue.

@FlorianHeigl
Copy link

@ghasemloo ty for this, saving a many hour detour

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.22
Development

No branches or pull requests
4 participants
@Snaipe @FlorianHeigl @bcmills @ghasemloo