Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go,x/website: GOPROXY=direct go list -versions -m go is missing most versions #61359

Closed
bcmills opened this issue Jul 14, 2023 · 9 comments
Closed

cmd/go,x/website: GOPROXY=direct go list -versions -m go is missing most versions #61359

bcmills opened this issue Jul 14, 2023 · 9 comments
Assignees
@suzmue
Labels
GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. Suggested Issues that may be good for new contributors looking for work to do. website
Milestone
Go1.22

Comments

@bcmills
Copy link
Contributor

bcmills commented Jul 14, 2023

What version of Go are you using (go version)?

~$ go1.21rc2 version
go version go1.21rc2 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

What did you do?

GOPROXY=direct go1.21rc2 list -m -versions go

What did you expect to see?

The same results as with GOPROXY=https://proxy.golang.org.

What did you see instead?

~$ GOPROXY=direct go1.21rc2 list -m -versions go
go 1.19.11 1.20.6

~$ GOPROXY=https://proxy.golang.org go1.21rc2 list -m -versions go
go 1.9.2rc2 1.2.2 1.3rc1 1.3rc2 1.3 1.3.1 1.3.2 1.3.3 1.4beta1 1.4rc1 1.4rc2 1.4 1.4.1 1.4.2 1.4.3 1.5beta1 1.5beta2 1.5beta3 1.5rc1 1.5 1.5.1 1.5.2 1.5.3 1.5.4 1.6beta1 1.6beta2 1.6rc1 1.6rc2 1.6 1.6.1 1.6.2 1.6.3 1.6.4 1.7beta1 1.7beta2 1.7rc1 1.7rc2 1.7rc3 1.7rc4 1.7rc5 1.7rc6 1.7 1.7.1 1.7.3 1.7.4 1.7.5 1.7.6 1.8beta1 1.8beta2 1.8rc1 1.8rc2 1.8rc3 1.8 1.8.1 1.8.2 1.8.3 1.8.4 1.8.5 1.8.6 1.8.7 1.9beta1 1.9beta2 1.9rc1 1.9rc2 1.9 1.9.1 1.9.2 1.9.3 1.9.4 1.9.5 1.9.6 1.9.7 1.10beta2 1.10rc1 1.10rc2 1.10 1.10.1 1.10.2 1.10.3 1.10.4 1.10.5 1.10.6 1.10.7 1.10.8 1.11beta1 1.11beta2 1.11beta3 1.11rc1 1.11rc2 1.11 1.11.1 1.11.2 1.11.3 1.11.4 1.11.5 1.11.6 1.11.7 1.11.8 1.11.9 1.11.10 1.11.11 1.11.12 1.11.13 1.12beta1 1.12beta2 1.12rc1 1.12 1.12.1 1.12.2 1.12.3 1.12.4 1.12.5 1.12.6 1.12.7 1.12.8 1.12.9 1.12.10 1.12.11 1.12.12 1.12.13 1.12.14 1.12.15 1.12.16 1.12.17 1.13beta1 1.13rc1 1.13rc2 1.13 1.13.1 1.13.2 1.13.3 1.13.4 1.13.5 1.13.6 1.13.7 1.13.8 1.13.9 1.13.10 1.13.11 1.13.12 1.13.13 1.13.14 1.13.15 1.14beta1 1.14rc1 1.14 1.14.1 1.14.2 1.14.3 1.14.4 1.14.5 1.14.6 1.14.7 1.14.8 1.14.9 1.14.10 1.14.11 1.14.12 1.14.13 1.14.14 1.14.15 1.15beta1 1.15rc1 1.15rc2 1.15 1.15.1 1.15.2 1.15.3 1.15.4 1.15.5 1.15.6 1.15.7 1.15.8 1.15.9 1.15.10 1.15.11 1.15.12 1.15.13 1.15.14 1.15.15 1.16beta1 1.16rc1 1.16 1.16.1 1.16.2 1.16.3 1.16.4 1.16.5 1.16.6 1.16.7 1.16.8 1.16.9 1.16.10 1.16.11 1.16.12 1.16.13 1.16.14 1.16.15 1.17beta1 1.17rc1 1.17rc2 1.17 1.17.1 1.17.2 1.17.3 1.17.4 1.17.5 1.17.6 1.17.7 1.17.8 1.17.9 1.17.10 1.17.11 1.17.12 1.17.13 1.18beta1 1.18beta2 1.18rc1 1.18 1.18.1 1.18.2 1.18.3 1.18.4 1.18.5 1.18.6 1.18.7 1.18.8 1.18.9 1.18.10 1.19beta1 1.19rc1 1.19rc2 1.19 1.19.1 1.19.2 1.19.3 1.19.4 1.19.5 1.19.6 1.19.7 1.19.8 1.19.9 1.19.10 1.19.11 1.20rc1 1.20rc2 1.20rc3 1.20 1.20.1 1.20.2 1.20.3 1.20.4 1.20.5 1.20.6 1.21rc2 1.21rc3

This also reproduces with go1.20.6 list -versions -m golang.org/toolchain: GOPROXY=direct causes the list of available toolchains to be substantially reduced.
@bcmills bcmills added this to the Go1.21 milestone Jul 14, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Jul 14, 2023

Found via #61358.

@bcmills
Copy link
Contributor Author

bcmills commented Jul 14, 2023

A bit of debugging:

~$ curl -s 'https://golang.org/toolchain?go-get=1' | grep go-import
<meta name="go-import" content="golang.org/toolchain mod https://go.dev/dl/mod">

~$ curl 'https://go.dev/dl/mod/golang.org/toolchain/@v/list'
v0.0.1-go1.20.6.darwin-amd64
v0.0.1-go1.20.6.darwin-arm64
v0.0.1-go1.20.6.freebsd-386
v0.0.1-go1.20.6.freebsd-amd64
v0.0.1-go1.20.6.linux-386
v0.0.1-go1.20.6.linux-amd64
v0.0.1-go1.20.6.linux-arm64
v0.0.1-go1.20.6.linux-arm
v0.0.1-go1.20.6.linux-ppc64le
v0.0.1-go1.20.6.linux-s390x
v0.0.1-go1.20.6.windows-386
v0.0.1-go1.20.6.windows-amd64
v0.0.1-go1.20.6.windows-arm64
v0.0.1-go1.19.11.darwin-amd64
v0.0.1-go1.19.11.darwin-arm64
v0.0.1-go1.19.11.freebsd-386
v0.0.1-go1.19.11.freebsd-amd64
v0.0.1-go1.19.11.linux-386
v0.0.1-go1.19.11.linux-amd64
v0.0.1-go1.19.11.linux-arm64
v0.0.1-go1.19.11.linux-arm
v0.0.1-go1.19.11.linux-ppc64le
v0.0.1-go1.19.11.linux-s390x
v0.0.1-go1.19.11.windows-386
v0.0.1-go1.19.11.windows-amd64
v0.0.1-go1.19.11.windows-arm64

Maybe the go-import statement is supposed to point to https://proxy.golang.org instead?

(attn @rsc @golang/release)

@cherrymui cherrymui added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 14, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Jul 14, 2023

This appears to be intentional:
https://cs.opensource.google/go/x/website/+/refs/heads/master:internal/dl/server.go;l=84;drc=92acdba30850746b5b9a31fdae5d411987d6247e

Still, we should double-check the user experience with this configuration. (I had assumed that everything would go through — or be consistent with — proxy.golang.org.)

@gopherbot
Copy link

Change https://go.dev/cl/509637 mentions this issue: cmd/go: fix tests with GOPROXY=direct in go.env

@rsc
Copy link
Contributor

rsc commented Jul 15, 2023

Everything can't "go through" proxy.golang.org because proxy.golang.org has to fetch from somewhere, using GOPROXY=direct. That somewhere is go.dev/dl/mod. We should probably change /dl/mod to match the proxy and list all known versions.

gopherbot pushed a commit that referenced this issue Jul 17, 2023
@gopherbot
cmd/go: fix tests when go.env sets GOPROXY=direct GOSUMDB=off
229cde5 
Tested locally by changing GOROOT/go.env. At some point perhaps we
should also set up a builder that runs with some common expected
modifications to go.env
(such as GOTOOLCHAIN=local GOPROXY=direct GOSUMDB=off).

Fixes #61358.
Updates #61359.

Change-Id: I365ec536bec86370e302fb726fa897400ab42cf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/509637
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
@rsc
Copy link
Contributor

rsc commented Jul 18, 2023

If anyone wants to send a CL for x/website, please go ahead.

@dmitshur dmitshur added Suggested Issues that may be good for new contributors looking for work to do. NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 28, 2023
bradfitz pushed a commit to tailscale/go that referenced this issue Aug 2, 2023
@bradfitz
cmd/go: fix tests when go.env sets GOPROXY=direct GOSUMDB=off
b82e7d4 
Tested locally by changing GOROOT/go.env. At some point perhaps we
should also set up a builder that runs with some common expected
modifications to go.env
(such as GOTOOLCHAIN=local GOPROXY=direct GOSUMDB=off).

Fixes golang#61358.
Updates golang#61359.

Change-Id: I365ec536bec86370e302fb726fa897400ab42cf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/509637
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
@gopherbot gopherbot modified the milestones: Go1.21, Go1.22 Aug 8, 2023
@suzmue
Copy link
Contributor

suzmue commented Dec 19, 2023

@dmitshur and I discovered that this bug will prevent proxy.golang.org from downloading modules that require the latest unstable go toolchain before a higher stable version is released.

Will take a look at fixing this in the website.

@suzmue suzmue self-assigned this Dec 19, 2023
@gopherbot
Copy link

Change https://go.dev/cl/551595 mentions this issue: cmd/golangorg: return all known Go toolchain versions

@dmitshur
Copy link
Contributor

dmitshur commented Jan 3, 2024 •
edited

For posterity, noting that the earlier output of go list -m -versions go (with the default GOPROXY value) as seen in the original report was missing Go version 1.10beta1 for some reason, even though it's a pre-release version that exists (https://go.dev/dl/#go1.10beta1, announcement). Fixing this issue caused it to appear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@suzmue suzmue

Labels
GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. Suggested Issues that may be good for new contributors looking for work to do. website
Projects
None yet
Milestone
Go1.22
Development

No branches or pull requests
6 participants
@rsc @dmitshur @bcmills @suzmue @gopherbot @cherrymui