net/http: provide better error message when proxy is misconfigured #29809

jtfogarty · 2019-01-18T15:33:13Z

What version of Go are you using (`go version`)?

$ go version go1.11 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build200009662=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Doing a go get fails behind a corporate proxy if environment variables are set as below; HTTP_PROXY=http://proxyIp:proxyPort
it succeeds if set as
HTTP_PROXY=proxyIp:proxyPort
this is true for HTTP or HTTPS variables This is also true for any go project using net/http
I'm seeing this issue with Docker versions greater than 17.09, Jenkins-x, goBuffalo
Older versions of go (specifically 1.8.3 - the only version I tested) do not behave this way
Docker version 17.09 and lower do not behave this way.

What did you expect to see?

I expect it to pull through the proxy server with the below environment variables set;
HTTP_PROXY=http://proxyIp:proxyPort
HTTPS_PROXY=https://proxyIp:proxyPort

What did you see instead?

Process fails to get through the proxy.

The text was updated successfully, but these errors were encountered:

bradfitz · 2019-01-18T16:29:58Z

/cc @bcmills for cmd/go.

bcmills · 2019-01-22T19:30:21Z

What did you do?

Doing a go get fails […]

More detail, please.

Fails in what way? (Does the error message indicate what went wrong?)
You appear to be using an insecure (HTTP-only) proxy. Did you pass the -insecure flag to go get?

jtfogarty · 2019-01-22T20:37:41Z

With variables as below;

http_proxy=http://proxyIp:proxyPort
https_proxy=https://proxyIp:proxyPort

The error is;

go get golang.org/x/blog/content
package golang.org/x/blog/content: unrecognized import path "golang.org/x/blog/content" (https fetch: Get https://golang.org/x/blog/content?go-get=1: proxyconnect tcp: EOF)

With variables as;

http_proxy=proxyIp:proxyPort
https_proxy=proxyIp:proxyPort

The below succeeds
go get golang.org/x/blog/content

Not sure about your point on HTTP only. The go get command does not specify either.

bcmills · 2019-01-22T21:24:20Z

What happens if you run curl https://golang.org/x/blog/content?go-get=1 in the same environment?

jtfogarty · 2019-01-22T21:32:59Z

Works fine when variables are set to;

http_proxy=http://proxyIp:proxyPort
https_proxy=https://proxyIp:proxyPort

curl https://golang.org/x/blog/content?go-get=1
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<meta name="go-import" content="golang.org/x/blog git https://go.googlesource.com/blog">
<meta name="go-source" content="golang.org/x/blog https://github.com/golang/blog/ https://github.com/golang/blog/tree/master{/dir} https://github.com/golang/blog/blob/master{/dir}/{file}#L{line}">
<meta http-equiv="refresh" content="0; url=https://godoc.org/golang.org/x/blog/content">
</head>
<body>
Nothing to see here; <a href="https://godoc.org/golang.org/x/blog/content">move along</a>.
</body>
</html>

bradfitz · 2019-04-29T20:17:30Z

@jtfogarty, sorry for the delay here. I was actively ignoring cmd/go issues, the cmd/go owners deal with them, but since you asked me about this at GCP Next I'm taking a look.

There's a difference between:

https_proxy=https://proxyIp:proxyPort

and

https_proxy=proxyIp:proxyPort

The former proxies https requests (e.g. those from cmd/go get) using CONNECT requests to an https proxy server at the given ip:port.

The latter proxies those https requests still using CONNECT requests but to an http proxy server at the given ip:port.

The notable difference is that you almost certainly don't have a valid https certificate for your proxy server if you're connecting via an IP address and not via a hostname.

There are a few rare exceptions (like https://1.1.1.1/) but generally IP addresses don't get TLS certs, and only hostnames do.

Admittedly the error message above kinda sucks:

(https fetch: Get https://golang.org/x/blog/content?go-get=1: proxyconnect tcp: EOF)

It looks like your proxy is immediately hanging up on you (cmd/go) when it's not presented any SNI value (a hostname) in the TLS ClientHello.

I get a slightly better error if I use a dummy IP address for a proxy:

bradfitz@go:~$ GOROOT=$HOME/go GODEBUG=http2debug GO111MODULE=off HTTPS_PROXY=https://212.47.227.56:443 go get -u -v bradfitz.com/foo
Fetching https://bradfitz.com/foo?go-get=1
https fetch failed: Get https://bradfitz.com/foo?go-get=1: proxyconnect tcp: remote error: tls: internal error
package bradfitz.com/foo: unrecognized import path "bradfitz.com/foo" (https fetch: Get https://bradfitz.com/foo?go-get=1: proxyconnect tcp: remote error: tls: internal error)

There at least I see (proxyconnect tcp: remote error: tls: internal error)

But perhaps we could detect this case (an https proxy without a hostname) and upgrade the error message to something more helpful with a hint ("you need a hostname probably") along with the original error.

Can you confirm that's your issue?

jtfogarty · 2019-05-04T16:01:11Z

Hi @bradfitz We are connecting via a hostname. Sorry for the confusion.

bradfitz · 2019-05-07T03:03:41Z

Maybe your proxy doesn't listen on https.

What does curl say when you do curl -I https://ip:port ?

jtfogarty · 2019-05-07T03:39:32Z

This is only an issue for apps written in go. I was installing a server with helm today. I was able to execute the following https_proxy=https://ServerName:Port wget https://storage.googleapis.com/kubernetes-helm/helm-v2.13.1-linux-amd64.tar.gz .
and it downloaded fine. I attempted to use install a helm chart and I ran into the same issue. The fix was to execute https_proxy=ServerName:Port helm fetch https://charts.rook.io/release

bradfitz · 2019-05-07T05:16:25Z

What does curl say when you do curl -I https://ip:port ?

jtfogarty · 2019-05-07T17:21:59Z

Not sure what ip:port you want me to curl to but if I curl to github.com with the https_proxy variable set, I get the following;

curl -I https://www.github.com
HTTP/1.1 200 Connection established

HTTP/1.1 301 Moved Permanently
Content-length: 0
Location: https://github.com/

Using the ip for github

curl -I https://192.30.253.112
HTTP/1.1 403 Forbidden
Mime-Version: 1.0
Date: Tue, 07 May 2019 12:14:14 CDT
Via: 1.1 InternalServerName:Port (Cisco-WSA/10.5.2-061)
Content-Type: text/html
Connection: keep-alive
Proxy-Connection: keep-alive
Content-Length: 2844

curl: (56) Received HTTP code 403 from proxy after CONNECT

bradfitz · 2019-05-07T17:37:31Z

So I bet the issue here is that Go just can't validate your InternalServerName:Port's TLS cert.

I'm afraid I can't really debug further by talking in the abstract. I'd need more verbose logs that you'd inevitably need to redact too much to be useful.

If you want to privately email me the full details, I'm bradfitz at golang.org.

bradfitz · 2019-05-08T17:15:25Z

This ended up being an environment configuration problem.

User had:

https_proxy=https://foobar:1234
http_proxy=http://foobar:1234

Where foobar:1234 was only an HTTP server, not https. (It could proxy an https request via CONNECT, but it did not speak https itself)

openssl s_client and curl confirmed that foobar:1234 only spoke http.

Go could perhaps provide a better error message than EOF when this occurs, though.

bradfitz changed the title ~~net/http Proxy Failure~~ cmd/go: http proxy failure Jan 18, 2019

bradfitz added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 18, 2019

bradfitz added this to the Go1.13 milestone Jan 18, 2019

bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. GoCommand cmd/go labels Jan 22, 2019

bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 22, 2019

bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 22, 2019

agnivade removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 29, 2019

bcmills self-assigned this Feb 12, 2019

bradfitz changed the title ~~cmd/go: http proxy failure~~ cmd/go: bad error message when using https proxy by IP instead of hostname Apr 29, 2019

bradfitz unassigned bcmills May 8, 2019

bradfitz removed the GoCommand cmd/go label May 8, 2019

bradfitz changed the title ~~cmd/go: bad error message when using https proxy by IP instead of hostname~~ net/http: provide better error message when proxy is misconfigured May 8, 2019

bradfitz added help wanted NeedsFix The path to resolution is known, but the work has not been done. labels May 8, 2019

bradfitz modified the milestones: Go1.13, Unplanned May 8, 2019

gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 8, 2019

bcmills mentioned this issue May 15, 2021

proxy.golang.org: go mod vendor, go get <package>, etc not working with "unexpected EOF" error #46179

Closed

berk-can mentioned this issue Sep 24, 2021

go get gives EOF #48599

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: provide better error message when proxy is misconfigured #29809

net/http: provide better error message when proxy is misconfigured #29809

jtfogarty commented Jan 18, 2019

bradfitz commented Jan 18, 2019

bcmills commented Jan 22, 2019

What did you do?

jtfogarty commented Jan 22, 2019

bcmills commented Jan 22, 2019

jtfogarty commented Jan 22, 2019

bradfitz commented Apr 29, 2019

jtfogarty commented May 4, 2019

bradfitz commented May 7, 2019

jtfogarty commented May 7, 2019

bradfitz commented May 7, 2019

jtfogarty commented May 7, 2019

bradfitz commented May 7, 2019

bradfitz commented May 8, 2019

net/http: provide better error message when proxy is misconfigured #29809

net/http: provide better error message when proxy is misconfigured #29809

Comments

jtfogarty commented Jan 18, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bradfitz commented Jan 18, 2019

bcmills commented Jan 22, 2019

What did you do?

jtfogarty commented Jan 22, 2019

bcmills commented Jan 22, 2019

jtfogarty commented Jan 22, 2019

bradfitz commented Apr 29, 2019

jtfogarty commented May 4, 2019

bradfitz commented May 7, 2019

jtfogarty commented May 7, 2019

bradfitz commented May 7, 2019

jtfogarty commented May 7, 2019

bradfitz commented May 7, 2019

bradfitz commented May 8, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?