Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: ProxyFromEnvironment does not support socks5h #24135

Closed
sorsasampo opened this issue Feb 26, 2018 · 8 comments
Closed

net/http: ProxyFromEnvironment does not support socks5h #24135

sorsasampo opened this issue Feb 26, 2018 · 8 comments
Labels
FeatureRequest help wanted NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@sorsasampo
Copy link

What version of Go are you using (go version)?

go version go1.10 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GORACE=""
GOROOT="/home/user/go1.10"
GOTMPDIR=""
GOTOOLDIR="/home/user/go1.10/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build712629011=/tmp/go-build -gno-record-gcc-switches"

What did you do?

What did you expect to see?

What did you see instead?

go get does not understand socks5h in HTTPS_PROXY, even though it understands socks5.

For example, to test using the tor proxy:

$ HTTPS_PROXY=socks5h://127.0.0.1:9050 go get -v golang.org/x/tools/cmd/goimports
Fetching https://golang.org/x/tools/cmd/goimports?go-get=1
https fetch failed: Get https://golang.org/x/tools/cmd/goimports?go-get=1: proxyconnect tcp: dial tcp: lookup socks5h: No address associated with hostname
package golang.org/x/tools/cmd/goimports: unrecognized import path "golang.org/x/tools/cmd/goimports" (https fetch: Get https://golang.org/x/tools/cmd/goimports?go-get=1: proxyconnect tcp: dial tcp: lookup socks5h: No address associated with hostname)

Other tools that understand HTTPS_PROXY also understand socks5h (resolve hostname through the proxy, not locally). curl and yum, for example.

Related to issue #18508.

@bradfitz bradfitz added help wanted NeedsFix The path to resolution is known, but the work has not been done. FeatureRequest labels Feb 26, 2018
@bradfitz bradfitz added this to the Unplanned milestone Feb 26, 2018
@bradfitz
Copy link
Contributor

Seems reasonable. We need to clean up the x/net/socks* packages first, but then somebody can do this.

/cc @mikioh

@gopherbot
Copy link

Change https://golang.org/cl/156518 mentions this issue: http/httpproxy: support socks5h scheme in proxy URL

@ChrMa
Copy link

ChrMa commented Sep 8, 2020

As a git user, which has recently started using git-lfs, which in turn is written in go, I've stumbled across the same issue.
Sometimes i need to use git with a socks5h HTTPS_PROXY, and git works just fine, but git-ls gives the above error message saying host socks5h cannot be resolved. When I set HTTPS_PROXY to using socks5 instead, git-ls works, but now git tries to resolve all host names and give the resulting IPs to the socks server, which does not work with my setup and is one of the reasons I'm using the socks server in the first place.
The only way around this is using a combination of HTTPS_PROXY=socks5://server:port and https_proxy=socks5h://server:port, which works because git-lfs gives HTTPS_PROXY precedence over https_proxy, while git itself does exactly the opposite.
Given that the socks5 implemented in go is in reality socks5h in the first place, why not add socks5h to the list of proxy protocols recognized and just handle it the same as socks5? The above mentioned change https://go-review.googlesource.com/c/net/+/156518 seems to try exactly this, even though I don't quite understand, why it split off part of the change into another change at https://go-review.googlesource.com/c/go/+/162318 .

@thatnerdjosh
Copy link

thatnerdjosh commented Dec 10, 2020

I volunteer to work on this support if no one is working on it

EDIT: @mikioh is the cleanup on the SOCKS module done yet?

/CC @bradfitz (if you still are a maintainer)

bekcpear added a commit to bekcpear/gentoo that referenced this issue Dec 29, 2022
Golang does not support the 'socks5h://' schema for http[s]_proxy
env variable: https://github.com/golang/go/blob/9123221ccf3c80c741ead5b6f2e960573b1676b9/src/vendor/golang.org/x/net/http/httpproxy/proxy.go#L152-L159,
while libcurl supports it: https://github.com/curl/curl/blob/ae98b85020094fb04eee7e7b4ec4eb1a38a98b98/docs/libcurl/opts/CURLOPT_PROXY.3#L48-L59.
So, if a 'https_proxy=socks5h://127.0.0.1:1080' env has been set in the
make.conf to make curl (assuming curl is the current download command) to
download all packages through the proxy, go-module_live_vendor will
fail.

The only difference between these two schemas is, 'socks5h' will solve
the hostname via the proxy while 'socks5' will not. I think it's ok to
fallback 'socks5h' to 'socks5' for `go vendor` command and warn user,
until golang supports it.

related to issue: golang/go#24135
bekcpear added a commit to bekcpear/gentoo that referenced this issue Dec 29, 2022
Golang does not support the 'socks5h://' schema for http[s]_proxy
env variable: https://github.com/golang/go/blob/9123221ccf3c80c741ead5b6f2e960573b1676b9/src/vendor/golang.org/x/net/http/httpproxy/proxy.go#L152-L159,
while libcurl supports it: https://github.com/curl/curl/blob/ae98b85020094fb04eee7e7b4ec4eb1a38a98b98/docs/libcurl/opts/CURLOPT_PROXY.3#L48-L59.
So, if a 'https_proxy=socks5h://127.0.0.1:1080' env has been set in the
make.conf to make curl (assuming curl is the current download command) to
download all packages through the proxy, go-module_live_vendor will
fail.

The only difference between these two schemas is, 'socks5h' will solve
the hostname via the proxy while 'socks5' will not. I think it's ok to
fallback 'socks5h' to 'socks5' for `go vendor` command and warn user,
until golang supports it.

related to issue: golang/go#24135

Signed-off-by: Ryan Qian <i@bitbili.net>
huww98 added a commit to huww98/net that referenced this issue Sep 3, 2023
currently only http/https/socks5 scheme are allowed. However, any scheme could
be possible if user provides their own implementation. Specifically, the widely
used `socks5h://localhost` is parsed as Scheme="http" Host="socks5h:", which
does not make sense because host name cannot contain ":".

This patch allows any scheme to appear in the proxy config. And only fallback
to http scheme if parsed scheme or host is empty.

`url.Parse` result of fallback cases:

localhost      => Scheme="localhost"
localhost:1234 => Scheme="localhost" Opaque="1234"
example.com    => Path="example.com"

Updates golang/go#24135
@gopherbot
Copy link

Change https://go.dev/cl/525257 mentions this issue: httpproxy: allow any scheme

@huww98
Copy link
Contributor

huww98 commented Oct 30, 2023

Could someone help me review this cl https://go.dev/cl/525257 please?

gopherbot pushed a commit to golang/net that referenced this issue Mar 8, 2024
currently only http/https/socks5 scheme are allowed. However, any scheme
could be possible if user provides their own implementation.
Specifically, the widely used "socks5h://localhost" is parsed as
Scheme="http" Host="socks5h:", which does not make sense because host
name cannot contain ":".

This patch allows any scheme to appear in the proxy config. And only
fallback to http scheme if parsed scheme or host is empty.

url.Parse() result of fallback cases:

localhost      => Scheme="localhost"
localhost:1234 => Scheme="localhost" Opaque="1234"
example.com    => Path="example.com"

Updates golang/go#24135

Change-Id: Ia2c041e37e2ac61be16220fd41d6cb6fabeeca3d
Reviewed-on: https://go-review.googlesource.com/c/net/+/525257
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
huww98 added a commit to huww98/go that referenced this issue Mar 8, 2024
Extend the net/http Transport to recognize the 'socks5h' schema as an alias for 'socks5'. Traditionally, the 'socks5h' schema indicates that the hostname should be resolved by the proxy server, which is behavior already implemented in Go for 'socks5'.

The update requires upgrading the golang.org/x/net dependency to support any proxy schema, including 'socks5h', specified through environment variables.

Fixes golang#24135
@gopherbot
Copy link

Change https://go.dev/cl/569977 mentions this issue: net/http: support socks5h proxy schema

@gopherbot
Copy link

Change https://go.dev/cl/570156 mentions this issue: all: update golang.org/x/net

gopherbot pushed a commit that referenced this issue Mar 9, 2024
Commands run (in both src and src/cmd):
	go get golang.org/x/net@master
	go mod tidy
	go mod vendor

For #24135

Change-Id: I88084d174c15a65350be1b43e27de619dc6d4dd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/570156
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: 胡玮文 <huww98@outlook.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest help wanted NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

6 participants