net/http: with Transfer-Encoding: chunked, Transport doesn't reuse the tcp connection #15703
Comments
|
More interesting is that removing the redundant "Connection: keep-alive" header (which is implicit with HTTP/1.1) seems to matter. It usually (also odd) works when you don't set that header. Related previous stuff: rev 18072ad and #14867 and google/go-github#317 |
bradfitz
changed the title
|
Actually, I had misread your repro code the first time. This is working as intended. The lack of connection of reuse (that I see 40-60% of the time with your repo) isn't due to chunked encoding, but because the handler hasn't yet finished when you call If you put a And if you move the The race you're seeing is because depending on goroutine scheduling. You have two goroutines, S(erver) and C(lient):
You're seeing them sometimes scheduled like this:
That is a valid result. I recommend you remove the flush. I've added a new test in https://golang.org/cl/23200 to guarantee the existing behavior when the flush is in the right spot. Please let me know if I misunderstand something. |
|
CL https://golang.org/cl/23200 mentions this issue. |
Verify that for a server doing chunked encoding, with the final data and EOF arriving together, the client will reuse the connection even if it closes the body without seeing an EOF. The server sends at least one non-zero chunk and one zero chunk. This verifies that the client's bufio reading reads ahead and notes the EOF, so even if the JSON decoder doesn't read the EOF itself, as long as somebody sees it, a close won't forcible tear down the connection. This was true at least of https://golang.org/cl/21291 No code change. Test already passed (even with lots of runs, including in race mode with randomized goroutine scheduling). Updates #15703 Change-Id: I2140b3eec6b099b6b6e54f153fe271becac5d949 Reviewed-on: https://go-review.googlesource.com/23200 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
With both below versions:
go version go1.6.2 linux/amd64devel +b66b97e Mon May 16 15:05:04 2016 +0000,On Arch Linux, go env:
With this program https://play.golang.org/p/HHqlKiVR4o
I expect that http client reuses the tcp connection for first and second http request.
The problem is describe more deeply here:
Example : Ensure that http connection will be re-use go-kit/kit#249
Sometime, the http client doesn't reuse the tcp connection between first and second http request (I check it with Wireshark filter on port given into the log e.g. tcp.port == 35068).
When the tcp connection is not reused, delve shows me that (https://github.com/golang/go/blob/master/src/net/http/internal/chunked.go#L61)
cr.r.Buffered()return 0, and EOF isn't read (https://github.com/golang/go/blob/master/src/net/http/internal/chunked.go#L56), as the last 0 that indicates the end of chunked response isn't read.So http client send a tcp reset to http server and open a new tcp connection.
If I drain the response.Body (with
io.Copy(ioutil.Discard, resp.Body)), the tcp connection is always reused by the http client (even iscr.r.Buffered()returns 0)The text was updated successfully, but these errors were encountered: