Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: install cannot find non-module version when module proxy doesn't implement @latest #44594

Closed
mikesep opened this issue Feb 24, 2021 · 4 comments
Closed

cmd/go: install cannot find non-module version when module proxy doesn't implement @latest #44594

mikesep opened this issue Feb 24, 2021 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Unplanned

Comments

@mikesep
Copy link

mikesep commented Feb 24, 2021 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.16 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
...
GOARCH="amd64"
...
GOOS="darwin"
...

What did you do?

go install golang.org/x/lint/golint@latest

What did you expect to see?

Successful installation.

What did you see instead?

$ go install golang.org/x/lint/golint@latest
go install golang.org/x/lint/golint@latest: no matching versions for query "latest"

Description

I'm having trouble getting Go 1.15/1.16 to fetch a non-module package version when a module proxy that doesn't support $base/$module/@latest is in the list.

Querying my local module proxy for $goproxy/golang.org/x/lint/@v/list returns 200 OK with an empty response body. This is the same as https://proxy.golang.org/golang.org/x/lint/@v/list -- in fact, my local proxy is relaying the request to proxy.golang.org.

My local module proxy doesn't support $goproxy/golang.org/x/lint/@latest and returns 404 Not Found. proxy.golang.org returns 200 OK with a JSON body containing a pseudo-version.

The problem is that go doesn't seem to be moving on to try direct in my list of proxies. Per https://golang.org/ref/mod#goproxy-protocol, $base/$module/@latest isn't required to be implemented, but this combination of (list=>200, latest=>404) doesn't seem to be working with the current code.

Investigation so far

I've traced things through in a local build of go1.16 and found that execution is getting to the call to p.latest() on line 376:

go/src/cmd/go/internal/modfetch/proxy.go

Lines 370 to 377 in f21be2f

func (p *proxyRepo) Latest() (*RevInfo, error) {
data, err := p.getBytes("@latest")
if err != nil {
if !errors.Is(err, fs.ErrNotExist) {
return nil, p.versionError("", err)
}
return p.latest()
}

This call is a bit strange to me -- if querying @latest fails, it seems to be trying @v/list again, which it already did in an earlier call to proxyRepo.Versions(). I'm curious to know if it serves a purpose.

Even if I bypass the call to p.latest() in a local build and return nil, err right before that, I think it's going to

go/src/cmd/go/internal/modload/query.go

Lines 227 to 236 in f21be2f

if qm.mayUseLatest {
latest, err := repo.Latest()
if err == nil {
if qm.allowsVersion(ctx, latest.Version) {
return lookup(latest.Version)
}
} else if !errors.Is(err, fs.ErrNotExist) {
return nil, err
}
}
which doesn't return an ErrNotExist and instead falls through to the end:

go/src/cmd/go/internal/modload/query.go

Line 246 in f21be2f

return nil, &NoMatchingVersionError{query: query, current: current}

Since NoMatchingVersionError is not an ErrNotExist, modfetch.TryProxies sees this as an error and does not fail over to the next proxy in the list:

go/src/cmd/go/internal/modfetch/proxy.go

Lines 179 to 181 in f21be2f

if !proxy.fallBackOnError && !isNotExistErr {
break
}

I think my analysis here is right, but I'm not entirely sure of what all the surrounding code is doing, so I'm having trouble telling if Go is doing something wrong or I'm falling into a subtle but expected behavior.

My current idea for a fix would be to replace p.latest with simply returning the error and to return earlier in modload.queryProxy if repo.Latest() returns ErrNotExist. I haven't started testing this yet, and I'm putting in this Issue in hopes that someone can to let me know if this sounds right before I start going too far down the wrong path. Thanks!
@mikesep
Copy link
Author

mikesep commented Feb 24, 2021

See also: CL 183845 "cmd/go/internal/modfetch: halt proxy fallback if the proxy returns a non-404/410 response for @latest"

@bcmills
Copy link
Contributor

bcmills commented Feb 25, 2021

@jayconrod and @hyangah might have more insight, but I think this is probably working as designed (and as documented).

Since the @latest endpoint returns code 404, the go command interprets that as “@latest is not implemented”, and treats @v/list as authoritative. @v/list returns code 200, which confirms that the returned list (which is empty) is indeed authoritative.

I think that if you want go get to work with @latest (as opposed to a specific version), then you need one of three things:

@mikesep
Copy link
Author

mikesep commented Feb 25, 2021

Thanks for taking a look @bcmills. Previously, we were proxying gocenter.io, which includes pseudo-versions in @v/list, so we saw a change in behavior when we moved to proxy.golang.org. Obviously proxying only some of the underlying module proxy's implementation can cause confusion and is not ideal. I'll look into whether we can change the local proxy to @latest requests.

@seankhliao seankhliao changed the title go install cannot find non-module version when module proxy doesn't implement @latest cmd/go: install cannot find non-module version when module proxy doesn't implement @latest Feb 25, 2021
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Feb 25, 2021
@bcmills bcmills added this to the Unplanned milestone Feb 25, 2021
@jayconrod
Copy link
Contributor

Sorry for the slow response. This seems like an edge case, but I think I agree with @bcmills: this is working as intended.

I basically think of @v/list and @latest as parts of the same thing, even though they're two separate requests. If a proxy implements them (returning 200 for @v/list at least), the go command treats that as authoritative and won't fall back to other proxies, even if there are no matching versions.

Closing this issue, but happy to discuss further, here or on Slack.

@golang golang locked and limited conversation to collaborators Mar 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@mikesep @jayconrod @bcmills @gopherbot