Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: tries to resolve github.com as a module #33351

Closed
myitcv opened this issue Jul 29, 2019 · 7 comments
Closed

cmd/go: tries to resolve github.com as a module #33351

myitcv opened this issue Jul 29, 2019 · 7 comments
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance
Milestone

Comments

@myitcv
Copy link
Member

myitcv commented Jul 29, 2019

What version of Go are you using (go version)?

$ go version
go version devel +c4ca60313e Sun Jul 28 18:09:57 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/tmp/tmp.4LfxYnl2lH"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/tmp.Byzg7Tw4cq/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build167955296=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Simply reproduced via the following (with a clean GOPATH):

$ go get github.com/myitcv/gobin@v0.1.11
go: finding github.com/myitcv/gobin v0.1.11
go: finding github.com/myitcv/gobin v0.1.11
go: finding github.com v0.1.11
go: finding github.com/myitcv v0.1.11
go get github.com/myitcv/gobin@v0.1.11: github.com/myitcv/gobin@v0.1.11: invalid version: unknown revision v0.1.11

v0.1.11 does not exist; that's expected.

What did you expect to see?

No attempt to resolve github.com or github.com/myitcv

What did you see instead?

As above.

cc @bcmills @jayconrod

@myitcv myitcv added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Jul 29, 2019
@jayconrod
Copy link
Contributor

Is the issue that we're trying to resolve github.com and github.com/myitcv specifically? It would be reasonable to add a special case for github.com, since we know any module will have at least two path components after the domain there. However, I don't think we have any special cases here, and I'd be hesitant to start adding them. It would be nice to keep the proxy protocol very simple.

Or is the issue that, having discovered github.com/myitcv/gobin is a module (without v0.1.11), we stop searching for other modules with prefixes of that path? This part may be working as intended. Anecdotally, I've heard of packages moved from a root module to a nested module (I think this was in github.com/hashicorp/vault). There are overlapping versions across those modules, and it would not be safe to stop the search early.

@bcmills
Copy link
Contributor

bcmills commented Jul 29, 2019

Note that avoiding those particular paths is just an optimization, and the requests are made in parallel so the optimization would have relatively little effect on overall latency.

(To my knowledge, there is nothing currently stopping GitHub from using <meta name="go-import" […]> tags to publish their own module at the root of github.com, or to allow users to publish their own modules at github.com/$USER.)

@bcmills bcmills added this to the Unplanned milestone Jul 29, 2019
@bcmills
Copy link
Contributor

bcmills commented Jul 29, 2019

There is an interesting implication for program encapsulation, in that a top-level github.com module could potentially provide packages that allow access to the internal paths of other modules hosted on github.com.

However, the presence of such a module would be relatively easy to detect: if it can be fetched through proxy.golang.org then it will be recorded in the public index (at index.golang.org), and if it cannot be fetched through proxy.golang.org then it will also not be recorded in the checksum database (and go1.13 will error out at that step).

(Besides, projects hosted on github.com are already trusting GitHub to serve unadulterated contents for their modules. Trusting GitHub not to serve a malicious top-level module doesn't seem like much of a stretch.)

@myitcv
Copy link
Member Author

myitcv commented Jul 29, 2019

Ah, it could be my misunderstanding. I thought I remembered cmd/go stopping at github.com/X/Y and not going any higher... hence why I raised the issue. But my memory might well be failing me!

@hyangah
Copy link
Contributor

hyangah commented Jul 29, 2019

@myitcv I believe this is the recent change in go1.13. For users like me who don't know the underlying go command implementation the message "go: finding github.com v0.1.11" will be quite surprising. Shouldn't the message be hidden by default?

@bcmills
Copy link
Contributor

bcmills commented Jul 30, 2019

@hyangah, that's more the purview of #26152. (See #26152 (comment) for my current thoughts on that.)

@bcmills
Copy link
Contributor

bcmills commented Sep 16, 2019

Since as far as we're aware the impact on overall latency is negligible, we don't plan to change this — I don't think it would even be worth the code complexity as a contributed change. Closing.

@bcmills bcmills closed this as completed Sep 16, 2019
@golang golang locked and limited conversation to collaborators Sep 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance
Projects
None yet
Development

No branches or pull requests

5 participants