Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sum.golang.org: should not respond with "410 Gone" #41986

Closed
mpalmer opened this issue Oct 15, 2020 · 9 comments
Closed

sum.golang.org: should not respond with "410 Gone" #41986

mpalmer opened this issue Oct 15, 2020 · 9 comments
Assignees
@hyangah
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. proxy.golang.org WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
proxy.golang.org/...

Comments

@mpalmer
Copy link

mpalmer commented Oct 15, 2020

What version of Go are you using (go version)?

$ go version
go version go1.15.2 linux/amd64

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/tmp/.cache/go-build"
GOENV="/tmp/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/backend/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build244541875=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Added an initial release (v0.1.0) to the module github.com/sol1/gormzerolog, ran go run . in a private codebase which uses that module.

What did you expect to see?

Something slightly less final and forever than "410 Gone".

What did you see instead?

An error which indicated that sum.golang.org had responded to a request for the gormzerolog module with "410 Gone".

Commentary

410 is a HTTP response code that means "access to the target resource is no longer available at the origin server and that this condition is likely to be permanent." (RFC7231, s6.5.9). In the case of sum.golang.org, it seems unlikely that the origin server knows, or has reason to believe, that the lack of a third-party module version is likely to be permanent. Indeed, only a few minutes after serving that 410 response, the sum database was able to serve checksums for that module version, indicating that the lack of access to the target resource was, indeed, not permanent.

Rather than a 410, requests for module checksums which do not refer to a known module version should provide a response of 404 Not Found. Again, per RFC7231, s6.5.9:

If the origin server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) ought to be used instead.
@hyangah
Copy link
Contributor

hyangah commented Oct 15, 2020

@mpalmer Thanks for reaching out. As briefly hinted in #31995 (comment), this is an implementation detail of our caching. We want to utilize HTTP caching and, unfortunately, some CDNs do not support 404 response caching yet. We are keeping an eye on this.

The primary consumer of sum.golang.org is the go command and that does not care whether it's 404 or 410.
Other than this unfortunate deviation from the HTTP spec, is there any problem caused by the use of 410?

cc @katiehockman @heschik

@toothrot toothrot modified the milestones: Backlog, Unreleased Oct 15, 2020
@toothrot toothrot added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Oct 15, 2020
@mpalmer
Copy link
Author

mpalmer commented Oct 17, 2020

Well, it confused the living heck out of me for a while, because the go command reported the "unexpected" error (despite it apparently being an expected and known error), so it sent me on quite a goose chase.

@bcmills
Copy link
Contributor

bcmills commented Oct 17, 2020

the go command reported the "unexpected" error (despite it apparently being an expected and known error)

What was the error it gave you? Perhaps we can improve the error message somehow.

@mpalmer
Copy link
Author

mpalmer commented Oct 17, 2020

Whilst I'm not in a position to reproduce the problem just at the moment, and the original error disappeared off the backscroll buffer due to an unfortunately large find output, to the best of my recollection it matched that given in #31995; that is: "unexpected status (<url>): 410 Gone". If 410 has a particular (non-standard) meaning to the go command, then it can't really be "unexpected" when a 410 arrives. At the very least, saying "such and such was not found", rather than "unexpected error" would be a good start. Something like "Could not find checksums for (description); this could be caused by delayed caching, or an incorrect module name or version." would be even more usefully descriptive.

@iand iand mentioned this issue Nov 24, 2020
Open
@kalipichandiarumugam

This comment was marked as off-topic.

Sign in to view
@mpalmer

This comment was marked as off-topic.

Sign in to view
@weberc2
Copy link

weberc2 commented Dec 26, 2021

What was the error it gave you? Perhaps we can improve the error message somehow.

@bcmills I just struck upon this. The error I encountered was:

go: downloading github.com/weberc2/auth v0.0.9
go get: github.com/weberc2/auth@v0.0.9: verifying module: github.com/weberc2/auth@v0.0.9: reading https://sum.golang.org/lookup/github.com/weberc2/auth@v0.0.9: 410 Gone
        server response: not found: github.com/weberc2/auth@v0.0.9: invalid version: unknown revision v0.0.9

In my case, I tagged my library but forgot to git push --tags before running go get github.com/weberc2/auth@v0.0.9 from my downstream module.

@hyangah
Copy link
Contributor

hyangah commented Jul 28, 2022 •
edited

@suzmue @findleyr @heschi According to the latest google cdn doc, 404 is classified as a cacheable status code. Should we flip to use 404?
https://cloud.google.com/cdn/docs/caching

@hyangah hyangah self-assigned this Aug 1, 2022
@hyangah
Copy link
Contributor

hyangah commented Aug 1, 2022

This is done.

$ curl -I -L https://sum.golang.org/lookup/example.com/mod@v1.0.0
HTTP/2 404 
...

@hyangah hyangah closed this as completed Aug 1, 2022
@golang golang locked and limited conversation to collaborators Aug 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@hyangah hyangah

Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. proxy.golang.org WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
proxy.golang.org/backlog
Development

No branches or pull requests
8 participants
@mpalmer @toothrot @weberc2 @hyangah @bcmills @suzmue @gopherbot @kalipichandiarumugam