Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/net/http2: empty request body sent by HTTP/2 client under high-throughput #51842

Open
en-vee opened this issue Mar 21, 2022 · 2 comments
Open
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@en-vee
Copy link

en-vee commented Mar 21, 2022

What version of Go are you using (go version)?

$ go version
go version go1.17.6 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Building for RHEL 7, on OS X MacOS Monterey 12.2.1

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/abc/Library/Caches/go-build"
GOENV="/Users/abc/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/abc/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/abc/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pj/by4fdgcd4z159jc_l9hdws_r0000gn/T/go-build3032146476=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Create an HTTP/2 client using the net/http2 transport as below (I am using non-TLS/SSL connection to the HTTP/2 server) :

httpTransport = &http2.Transport{
					AllowHTTP: true,
					DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
						return net.Dial(network, addr)
					},
					StrictMaxConcurrentStreams: true,
				}

Multiple goroutines using the same http client (which uses this above transport), then attempt to send an HTTP POST request to the server. The number of goroutines attempting could be anywhere between 1000 to 10000 per second.

What did you expect to see?

Single TCP connection in the netstat output

What did you see instead?

Hundreds of TCP connections especially when the rate of invoking the http client is more than about 2000 per second.
I understand that this is related to #13397 for which a fix has apparently made it's way since Go 1.6.
But I am not sure why the number of TCP connections keeps growing to the order of hundreds.

@en-vee
Copy link
Author

en-vee commented Mar 21, 2022

Ok, so it appears that I get a single TCP connection when I set the StrictMaxConcurrentStreams to true.
However, at higher rates e.g. 5000 requests per second, the request JSON body that is sent to the server is empty.
Not sure why that is happening either ?
In Wireshark, I can see that the HEADERS are sent from client to server, but this is followed by a RST_STREAM from client to server. There is no JSON payload that is being sent.

@en-vee en-vee changed the title net/http2 net/http2 : Seeing empty request body sent by HTTP/2 client under high-throughput Mar 21, 2022
@mknyszek mknyszek changed the title net/http2 : Seeing empty request body sent by HTTP/2 client under high-throughput x/net/http2 : Seeing empty request body sent by HTTP/2 client under high-throughput Mar 21, 2022
@mknyszek mknyszek added this to the Unreleased milestone Mar 21, 2022
@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 21, 2022
@mknyszek mknyszek changed the title x/net/http2 : Seeing empty request body sent by HTTP/2 client under high-throughput x/net/http2: seeing empty request body sent by HTTP/2 client under high-throughput Mar 21, 2022
@mknyszek mknyszek changed the title x/net/http2: seeing empty request body sent by HTTP/2 client under high-throughput x/net/http2: empty request body sent by HTTP/2 client under high-throughput Mar 21, 2022
@mknyszek
Copy link
Contributor

CC @neild

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

2 participants