Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: fallback to direct sum calculation for private modules #34344

Closed
vearutop opened this issue Sep 17, 2019 · 5 comments
Closed

cmd/go: fallback to direct sum calculation for private modules #34344

vearutop opened this issue Sep 17, 2019 · 5 comments
Labels
FrozenDueToAge

Comments

@vearutop
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/viacheslav.poturaev/Library/Caches/go-build"
GOENV="/Users/viacheslav.poturaev/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/viacheslav.poturaev/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/h7/7qg3nbt91bb6mgk2xtqqpwc40000gp/T/go-build118803844=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Having private repo available in github.com/private-org/lib and git configured to use token:

git config --global --add url."https://$GITHUB_TOKEN@github.com/private-org".insteadOf "https://github.com/private-org"

And having a clear environment without module cache and GO* env vars.
And not having a go.sum file.
I was building a new app for first time.

What did you expect to see?

I expect to see successful build.

What did you see instead?

Build failed with error message:

verifying github.com/private-org/lib@v0.5.38: github.com/private-org/lib@v0.5.38: reading https://sum.golang.org/lookup/github.com/private-org/lib@v0.5.38: 410 Gone

Build failed when I've set up GOPROXY=direct.
Build was successful after I've set up GOPRIVATE=github.com/private-org.

If GOSUMDB does not include an explicit URL, the URL is chosen by probing the GOPROXY URLs for an endpoint indicating support for the checksum database, falling back to a direct connection to the named database if it is not supported by any proxy.

I think build should not be failed for modules that were explicitly not found (404 for private repos) and especially if direct value is set for GOPROXY.
@vearutop vearutop changed the title cmd/go: fallback to direct sum calculation for provate modules cmd/go: fallback to direct sum calculation for private modules Sep 17, 2019
@FiloSottile
Copy link
Contributor

Ignoring the checksum database whenever it returns an unauthenticated 404 would undermine its security: any untrusted proxy could just serve fake contents, declare it can proxy the sumdb, and return a 404 for that version.

GOPRIVATE=github.com/private-org is the correct configuration for your use case, as far as I can tell.

@vearutop
Copy link
Contributor Author

@FiloSottile would not it be more appropriate so solve potential security issue authenticating 404 responses (so that no proxy can spoof it) rather than hard-failing on private modules?

@FiloSottile
Copy link
Contributor

There is no way for a sumdb mirror to provide 404s authenticated by the central log for arbitrary names without contacting the central log, and it would be very hard to make those 404s auditable and accountable like the rest of the tree. The ability of proxies to mirror the whole sumdb is an important design goal, because it allows proxies not to leak any information to the sumdb if they decide not to.

@vearutop
Copy link
Contributor Author

I think it is not that complicated. Authenticated 404 can be built on top of latest entry (referenced in payload) without being added to log.

Auditor could check if 404 is valid any time by using referenced state of log and resolved module name.

@FiloSottile
Copy link
Contributor

Auditors wouldn't get to see most 404s, because they are not part of the tree.

@golang golang locked and limited conversation to collaborators Sep 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@FiloSottile @vearutop @gopherbot