Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: can't handle Onion domains #42283

Closed
melroy89 opened this issue Oct 29, 2020 · 6 comments
Closed

net/http: can't handle Onion domains #42283

melroy89 opened this issue Oct 29, 2020 · 6 comments
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@melroy89
Copy link

melroy89 commented Oct 29, 2020 •
edited

What version of Go are you using (go version)?

go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/melroy/.cache/go-build"
GOENV="/home/melroy/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/melroy/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/melroy/Documents/drone/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build611413724=/tmp/go-build -gno-record-gcc-switches"

What did you do?

While using the Go net/http package, try to connect to an valid onion address.

In my case I'm using Drone a CI tool in Docker written in Golang. Drone tries via Oauth2 to connect to the Git server (eg. Gitea), but the POST request on http://mydomain.onion/login/oauth/access_token for Gitea fails: no suitable address found. See code lines below for more details what Drone is doing.

Usage example: Code line of HTTP package usage in Drone.
Usage example 2: Call to Oauth2 package
--> Code line of HTTP usage in Oauth2 package of Drone

What did you expect to see?

I would expect that Golang is able to connect to onion domains by using the http package, WITHOUT errors.

What did you see instead?

dial tcp: address xxxx.onion: no suitable address found

Output of Drone:

ERRO[0026] oauth: cannot exchange code: secret: Post "http://xxx.onion/login/oauth/access_token": dial tcp: address xxxx.onion: no suitable address found 
DEBU[0026] cannot authenticate user: Post "http://yyy.onion/login/oauth/access_token": dial tcp: address yyy.onion: no suitable address found 
DEBU[0026]                                               fields.time="2020-10-29T21:49:40Z" latency=1.611572ms method=GET remote="172.19.0.1:40842" request="/login?code=secret%3D&state=secret" request-id=secret

More reports

I'm not the only one suffering from this problem. Maybe not fully related, but somebody that tries to use go get by using a onion domain to fetch packages: https://www.reddit.com/r/techsupport/comments/981frd/need_help_with_golang_and_tor/
It could be touching the same code base in Go?

Thanks.

Regards,
Melroy van den Berg
@dmitshur dmitshur changed the title Golang HTTP package can't handle Onion domains net/http: can't handle Onion domains Oct 30, 2020
@dmitshur dmitshur added this to the Backlog milestone Oct 30, 2020
@dmitshur dmitshur added FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 30, 2020
@dmitshur
Copy link
Contributor

CC @odeke-em, @bradfitz, @empijei.

@odeke-em
Copy link
Member

odeke-em commented Oct 31, 2020 •
edited

Thank you for filing this issue/question @Danger89 and welcome to the Go project!
Thank you for the tag @dmitshur.

Technically, given that .onion addresses aren't actual DNS names, they require you to connect to Tor first and then the Tor protocol will handle the tunnelling and direction to the appropriate resource -- thus DNS requests are ALWAYS going to fail if you don't have an appropriate reverse proxy. If you have your Tor compatible proxy running, you can create a reverse proxy in Go code by https://golang.org/pkg/net/http/httputil/#NewSingleHostReverseProxy or just set in your environment HTTP_PROXY as per https://golang.org/pkg/net/http/#RoundTripper.

From Go's stand point, and from pretty much every language and browser, without Tor-enabled support, resolving such domains won't work. Here is a reference to Facebook describing the problem and them announcing their .onion site https://www.facebook.com/notes/protect-the-graph/making-connections-to-facebook-more-secure/1526085754298237/

For a reference to a deliberate decision not to resolve them unless there is a SOCK5 proxy, please see #13705.

I am going to close this issue as non-actionable in 1 day, but in case someone else wants to chime in, or please feel free to correct me.

@bradfitz
Copy link
Contributor

bradfitz commented Nov 1, 2020

I agree with Odeke. We don't plan to support Tor natively in the standard library.

@odeke-em odeke-em closed this as completed Nov 2, 2020
@melroy89
Copy link
Author

melroy89 commented Nov 3, 2020 •
edited

Thank you for filing this issue/question @Danger89 and welcome to the Go project!
Thank you for the tag @dmitshur.

For a reference to a deliberate decision not to resolve them unless there is a SOCK5 proxy, please see #13705.

@odeke-em You know that Tor runs a SOCK5 proxy by default, on port 9050, so that is available..? In fact Tor by default will NOT provide you any HTTP proxy only sock5 (eg socks5h://127.0.0.1:9050).

Does GO support socks5h schema just like git and curl does?

@melroy89
Copy link
Author

melroy89 commented Nov 3, 2020

Apparently it doesn't ;( proxyconnect tcp: dial tcp: lookup socks5h on <ip>: no such host

@melroy89
Copy link
Author

melroy89 commented Nov 3, 2020

Using HTTPTunnelPort 8118 in Torrc and HTTP_PROXY=https://127.0.0.1:8118 will result in: proxyconnect tcp: dial tcp <ip:port>: connect: connection refused. since I Tor only provides CONNECT proxy not a GET proxy.

@golang golang locked and limited conversation to collaborators Nov 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@bradfitz @melroy89 @dmitshur @odeke-em @gopherbot