Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go Get throwing disabled by GOPRIVATE/GONOPROXY error #35861

Closed
lafronzt opened this issue Nov 26, 2019 · 12 comments
Closed

Go Get throwing disabled by GOPRIVATE/GONOPROXY error #35861

lafronzt opened this issue Nov 26, 2019 · 12 comments

Comments

@lafronzt
Copy link

lafronzt commented Nov 26, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13.4 windows/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\MNP1LHG\AppData\Local\go-build
set GOENV=C:\Users\MNP1LHG\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=tfs.ups.com
set GONOSUMDB=tfs.ups.com
set GOOS=windows
set GOPATH=C:\Users\MNP1LHG\go
set GOPRIVATE=tfs.ups.com
set GOPROXY=nexusrmps.njrar.us.ups.com:8443/repository/goproxy-test/
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\MNP1LHG\AppData\Local\Temp\go-build106505266=/tmp/go-build -gno-record-gcc-switches

What did you do?

I tried to run a go get to retrive a local package hosted on an internal repo

go get -v tfs.ups.com/tfs/opt/P04A/_git/file-download
go get tfs.ups.com/tfs/opt/P04A/_git/file-download: disabled by GOPRIVATE/GONOPROXY

What did you expect to see?

I expected to see go get output display the no go-import meta tags error.

What did you see instead?

Instead, I received the error above disabled by GOPRIVATE/GONOPROXY

Additional Info

If I set GOPROXY='direct', it seems to throw the correct no go-import meta tags error.

@jayconrod
Copy link
Contributor

This seems to be working correctly: GOPROXY is set to nexusrmps.njrar.us.ups.com:8443/repository/goproxy-test/, which instructs the go command to only download modules from that proxy. But GOPRIVATE (and GONOPROXY, which defaults to GOPRIVATE) is set to tfs.ups.com, which instructs the go command not to use a proxy for modules with the prefix tfs.ups.com. There's no place it's allowed to get those modules.

You may want to clear GOPRIVATE and set GONOSUMDB to tfs.ups.com. Or alternatively set GOPROXY to nexusrmps.njrar.us.ups.com:8443/repository/goproxy-test/,direct (adding direct to the end).

go help modules-private has more information on these settings.

@lafronzt
Copy link
Author

Ah okay, thank you so much @jayconrod.

@lafronzt
Copy link
Author

@jayconrod Here is the problem, I am noticing. If I add the direct, it allows go get to download packages from all sites, our use case is not to allow that. In theory, shouldn't the GONOPROXY set via the GOPRIVATE env allow for direct connections to that host only?

@jayconrod
Copy link
Contributor

@lafrontz So if I understand correctly, you want to download modules starting with tfs.ups.com directly from version control, and all other modules from your proxy, not from version control?

If that's correct, use the settings below. The proxy should return 404 or 410 responses for modules that can be retrieved directly. It should return 200 for modules it can provide and other 4xx codes (for example 403) for modules that should not be retrieved directly. The go command will only fall back to later sources in the GOPROXY list after a 404 or 410 response. GONOPROXY will cause it to skip all proxies for matching modules.

GOPROXY=nexusrmps.njrar.us.ups.com:8443/repository/goproxy-test/,direct
GOPRIVATE=tfs.ups.com

Alternatively, the proxy can be responsible for serving all allowed modules, and the go command can be forbidden from connecting to other sources.

GOPROXY=nexusrmps.njrar.us.ups.com:8443/repository/goproxy-test/
GONOSUMDB=tfs.ups.com

@lafronzt
Copy link
Author

lafronzt commented Nov 27, 2019

@jayconrod Let me explain some more facts in regards to our network
Our network does not allow direct access to the internet unless you are authenticated and connecting through an HTTP Proxy server.
The Go Proxy hosted at nexusrmps.njrar.us.ups.com:8443 can connect to the internet since the Nexus server has credentials to connect to the internet.

In theory, we are trying to set it up, a user can only download Go packages via the Proxy, and the internal VCS (tfs.ups.com)

From the test, I have been running, if I add directly to the end of the GOPROXY env go get attempts to connect to the remote server directly. When running go get -v github.com/grailbio/bigslice

This output is given:
go get -v github.com/grailbio/bigslice go: finding github.com/grailbio/bigslice latest go: downloading github.com/grailbio/bigslice v0.0.0-20191127055513-215d6078cbbc verifying github.com/grailbio/bigslice@v0.0.0-20191127055513-215d6078cbbc: github.com/grailbio/bigslice@v0.0.0-20191127055513-215d6078cbbc: Get https://sum.golang.org/lookup/github.com/grailbio/bigslice@v0.0.0-20191127055513-215d6078cbbc: dial tcp 172.217.6.209:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

The IP Address in the response does not match the sum.golang.org one, nor any other internal network IPs.

@jayconrod
Copy link
Contributor

Sounds like it's hitting an error connecting to sum.golang.org, which is blocked on your network. You'll need to set GOSUMDB=off to disable it. (I'm not seeing the same IP address, but it is the same subnet).

Please look through go help module-private and module authentication failures for more info.

@lafronzt
Copy link
Author

lafronzt commented Nov 27, 2019

@jayconrod

Two things.

First, sum.golang.org is not that IP address; this is confusing for me.

nslookup sum.golang.org 1.1.1.1
Server:         1.1.1.1
Address:        1.1.1.1#53

Non-authoritative answer:
Name:   sum.golang.org
Address: 172.217.9.81
Name:   sum.golang.org
Address: 2607:f8b0:4009:804::2011

Second, according to the documentation, setting the GOPRIVATE which intern sets the GONOPROXY should skip the GOPROXY settings for private packages.

The GOPRIVATE and GONOPROXY environment variables allow bypassing the proxy for selected modules. See 'go help module-private' for details. - Module downloading and verification

The GOPRIVATE environment variable controls which modules the go command considers to be private (not available publicly) and should therefore not use the proxy or checksum database. Module configuration for non-public modules

The documentation goes on to say the following:

For example, if a company ran a module proxy serving private modules, users would configure go using:

GOPRIVATE=*.corp.example.com
GOPROXY=proxy.example.com
GONOPROXY=none

This would tell the go command and other tools that modules beginning with a corp.example.com > subdomain are private but that the company proxy should be used for downloading both public and private modules, because GONOPROXY has been set to a pattern that won't match any modules, overriding GOPRIVATE.

Which from how I am reading this, is that my original set up should have skipped the GOPROXY settings for tfs.ups.com.

set GONOPROXY=tfs.ups.com
set GONOSUMDB=tfs.ups.com
set GOPRIVATE=tfs.ups.com
set GOPROXY=nexusrmps.njrar.us.ups.com:8443/repository/goproxy-test/
set GOSUMDB=sum.golang.org

If I am misunderstanding this all, please explain it to me, because it does not seem to be adding up correctly.

@lucas-dehandschutter
Copy link

@lafronzt Did you find a solution for that problem?
I'm actually facing the same issue.

@lafronzt
Copy link
Author

@lucas-dehandschutter I do not seem to be having this issue anymore; however, I have had to set GOSUMDB=off since it is unavailable behind the proxy.

What exactly is the issue you are getting right now?

@hotshot-dot
Copy link

@lafronzt
Hello, I have browsed the above records, the problems I encountered are the same as yours, and even our needs are the same. Excuse me, how did you solve it,
thank you

@lafronzt
Copy link
Author

lafronzt commented Apr 2, 2020

@hotshot-dot it was actually never fixed.

my pending question to @jayconrod has never been addressed, and I am still noticing discrepancies between what they mentioned and what is written in the documentation.

@hotshot-dot
Copy link

@lafronzt
My problem has been solved.
The strange thing is that I did n’t use GOPRIVATE. Now, I can freely introduce private projects on gitlab, not as complicated as the official documentation.

@golang golang locked and limited conversation to collaborators Apr 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants