Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: Request to certain server hangs, works fine with curl #21301

Closed
wjdp opened this issue Aug 3, 2017 · 6 comments
Closed

net/http: Request to certain server hangs, works fine with curl #21301

wjdp opened this issue Aug 3, 2017 · 6 comments
Labels
FrozenDueToAge

Comments

@wjdp
Copy link

wjdp commented Aug 3, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8.3 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/will/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/will/go-build795220856=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

package main

import (
	"fmt"
	"net/http"
	"time"
)

func main() {
	client := &http.Client{
		Timeout: 30 * time.Second,
	}
	resp, err := client.Get("https://www.southampton.ac.uk/")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp)
}

What did you expect to see?

A response within a second or two

What did you see instead?

Timeout after 30 seconds, will happily hang for longer if timeout extended.

The following is the output of curl on the same machine / network. This timeout issue with this server has also been observed on a Travis test runner.

~ ❯ curl -v https://www.southampton.ac.uk/
*   Trying 152.78.118.52...
* Connected to www.southampton.ac.uk (152.78.118.52) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 697 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.southampton.ac.uk (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: OU=Domain Control Validated,CN=www.southampton.ac.uk
* 	 start date: Tue, 13 Jan 2015 00:00:00 GMT
* 	 expire date: Fri, 12 Jan 2018 23:59:59 GMT
* 	 issuer: C=NL,ST=Noord-Holland,L=Amsterdam,O=TERENA,CN=TERENA SSL CA 2
* 	 compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: www.southampton.ac.uk
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 03 Aug 2017 21:38:44 GMT
< Content-Type: text/html;charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: JSESSIONID=9E437837C159A9276F8DA64CFF182DE7; Path=/; HttpOnly
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Referrer-Policy: unsafe-url
< Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-inline' *.atbar.org *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk corporate.wordpress.soton.ac.uk blog.soton.ac.uk; script-src 'self' 'unsafe-inline' 'unsafe-eval' code.jquery.com www.googleadservices.com www.googletagmanager.com *.google-analytics.com connect.facebook.net static.ads-twitter.com j.flxpxl.com hj.flxpxl.com d2hlpp31teaww3.cloudfront.net d1d7fjtb6d4i2m.cloudfront.net analytics.twitter.com ib.adnxs.com www.intelliworkschat.com access.ecs.soton.ac.uk *.atbar.org www.youtube.com s.ytimg.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk stats.wp.com soton.disqus.com stats.wp.com corporate.wordpress.soton.ac.uk blog.soton.ac.uk s0.wp.com use.typekit.net connect.facebook.com platform.twitter.com; font-src 'self' data: ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk use.typekit.net; img-src data: 'self' www.southampton.ac.uk cdn.soton.ac.uk cdn.southampton.ac.uk t.co mpp2.vindicosuite.com *.google-analytics.com *.facebook.com pixel.adsafeprotected.com *.doubleclick.net www.google.com www.google.co.uk *.atbar.org *.issuu.com image.isu.pub www.intelliworkschat.com pixel.wp.com blog.soton.ac.uk platform.twitter.com pbs.twimg.com syndication.twitter.com p.typekit.net d1d7fjtb6d4i2m.cloudfront.net; frame-src 'self' www.google.com www.google.co.uk www.youtube.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk; child-src 'self' www.google.com www.google.co.uk www.youtube.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk;
< Set-Cookie: BIGipServer~CorpWeb~pool-corpweb-prod-80=2678630592.20480.0000; path=/; Httponly; Secure
< 
<!DOCTYPE html>
@bradfitz
Copy link
Contributor

bradfitz commented Aug 3, 2017

This is a dup of #20979

The server negotiates "h2" and has "BIGipServer" in its response headers.

I can verify with GODEBUG=http2debug that it does the immediate GOAWAY thing. Closing this bug and moving to #20979

@bradfitz bradfitz closed this as completed Aug 3, 2017
@bradfitz bradfitz mentioned this issue Aug 3, 2017
Open
@wjdp
Copy link
Author

wjdp commented Aug 3, 2017

Apologies for the duplicate and thanks for the quick response 😄

@odeke-em
Copy link
Member

odeke-em commented Aug 3, 2017

Oops, nvm @bradfitz beat me to it. @wjdp if you want to use http1, please use

package main

import (
	"crypto/tls"
	"flag"
	"fmt"
	"net/http"
	"time"
)

func main() {
	var http1 bool
	flag.BoolVar(&http1, "http1", false, "use http1 to invoke the server")
	flag.Parse()

	tr := &http.Transport{}
	if http1 {
		tr.TLSClientConfig = &tls.Config{
			NextProtos: []string{"h1"},
		}
	}

	client := &http.Client{
		Timeout:   30 * time.Second,
		Transport: tr,
	}
	resp, err := client.Get("https://www.southampton.ac.uk/")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp)
}

and invoke it with flag http1 ie go run main.go --http1 and that should return your result ASAP.

@wjdp
Copy link
Author

wjdp commented Aug 3, 2017

Thanks @odeke-em, provided snippet does the trick!

@mikioh mikioh changed the title net/http Request to certain server hangs, works fine with curl net/http: Request to certain server hangs, works fine with curl Aug 3, 2017
@HaraldNordgren
Copy link
Member

HaraldNordgren commented Jul 26, 2018 •
edited

I had a problem with a hanging host as well. On the surface very similar to this case although setting the protocol version did not work.

For me it was necessary in the TLS config to use

	DynamicRecordSizingDisabled: true

I posted a more fleshed out comment about it in: #20979 (comment) for debugging similar issues.

@bradfitz
Copy link
Contributor

@HaraldNordgren, please file a separate crypto/tls bug for your case. It sounds like a crypto/tls issue instead of a net/http issue.

FalseChord pushed a commit to FalseChord/readr-restful that referenced this issue Oct 12, 2018
Solve problem that sending "get" http request to ettoday will cause t…
4f5de4f 
…imeout.

1. Adcording to golang/go#21301, we use http1 in transport layer to prevent timeout.
2. Set meta parser timeout to 5 second, this prevent API blocking.
@golang golang locked and limited conversation to collaborators Sep 18, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@bradfitz @wjdp @odeke-em @gopherbot @HaraldNordgren