Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: RoundTripper is not sending tcp fin packet after response body close, even on exit #41342

Closed
mikumiku2468 opened this issue Sep 11, 2020 · 6 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@mikumiku2468
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.14"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.14/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build841420152=/tmp/go-build -gno-record-gcc-switches

go env Output
$ go env

What did you do?

package main

import (
"net/http"
"io/ioutil"
)

//var a string

func main() {
conntransport := &http.Transport{
DisableKeepAlives: true,
DisableCompression: true,
}
client := &http.Client{Transport: conntransport}
req, _ := http.NewRequest("GET", "http://10.255.254.161", nil)
req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36")
resp,_:=client.Do(req)
ioutil.ReadAll(resp.Body)
defer resp.Body.Close()
}

What did you expect to see?

I expect to see the tcp fin packet or tcp rst packet send after I use resp.Body.Close()

What did you see instead?

after I recv response from server
server send tcp fin psh ack flag packet
and I send tcp fin ack flag packet
and server send tcp ack packet
and client not send tcp fin or tcp rst packet

@mikumiku2468
Copy link
Author

I forgot to say
replace defer resp.Body.Close() by resp.Body.Close()
still not send fin flag packet

@toothrot toothrot changed the title golang http client do request not send tcp fin packet after response body close even program close net/http: RoundTripper is not sending tcp fin packet after response body close, even on exit Sep 14, 2020
@toothrot
Copy link
Contributor

I am not able to reproduce this locally, or I am missing something about your issue. The socket looks to be successfully closing to me.

8	0.000305806	::1	::1	TCP	86	8080 → 60916 [FIN, ACK] Seq=144 Ack=92 Win=65536 Len=0 TSval=3135899289 TSecr=3135899289
9	0.000374202	::1	::1	TCP	86	60916 → 8080 [FIN, ACK] Seq=92 Ack=145 Win=65536 Len=0 TSval=3135899289 TSecr=3135899289

As an aside, I believe DisableKeepAlives only applies to HTTP keep alives, not TCP keep alives. Regardless, I thought the FIN would be sent when the socket was closed, which I thought happened at the worst on program exit.

Does calling conntransport.CloseIdleConnections() do anything? How are you capturing packets?

@toothrot toothrot added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 14, 2020
@toothrot toothrot added this to the Backlog milestone Sep 14, 2020
@mikumiku2468
Copy link
Author

Hi toothrot
I add conntransport.CloseIdleConnections() and still not send fin packet

10.255.254.161 is server and 10.0.2.15 is client
tcpdump command
tcpdump -i any host 10.255.254.161 -n
here is tcpdump tail log
09:22:19.636000 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 10647:10844, ack 188, win 65535, length 197: HTTP
09:22:19.636027 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 10844, win 197, length 0
09:22:19.636257 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [FP.], seq 10844:10976, ack 188, win 65535, length 132: HTTP
09:22:19.636330 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [F.], seq 188, ack 10977, win 197, length 0
09:22:19.636533 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], ack 189, win 65535, length 0

for HTTP disablekeepalive is true since I want use 1 socket(tcp connection) per http request
so I except if server send response and send fin packet
client should send fin-ack then send fin packet
but client just send ack

tcpdump all log
09:22:19.487006 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [S], seq 3123866513, win 197, options [mss 1460,sackOK,TS val 692970230 ecr 0,nop,wscale 2], length 0
09:22:19.527305 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [S.], seq 192000001, ack 3123866514, win 65535, options [mss 1460], length 0
09:22:19.527330 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 1, win 197, length 0
09:22:19.527555 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [P.], seq 1:188, ack 1, win 197, length 187: HTTP: GET / HTTP/1.1
09:22:19.527881 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], ack 188, win 65535, length 0
09:22:19.569239 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 1:198, ack 188, win 65535, length 197: HTTP: HTTP/1.1 200 OK
09:22:19.569254 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 198, win 0, length 0
09:22:19.569415 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 198, win 197, length 0
09:22:19.569717 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 198:395, ack 188, win 65535, length 197: HTTP
09:22:19.569818 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 395, win 197, length 0
09:22:19.569965 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 395:592, ack 188, win 65535, length 197: HTTP
09:22:19.570054 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 592, win 197, length 0
09:22:19.570212 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 592:789, ack 188, win 65535, length 197: HTTP
09:22:19.570262 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 789, win 197, length 0
09:22:19.570493 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 789:986, ack 188, win 65535, length 197: HTTP
09:22:19.570573 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 986, win 197, length 0
09:22:19.570798 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 986:1183, ack 188, win 65535, length 197: HTTP
09:22:19.570913 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 1183, win 197, length 0
09:22:19.571164 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 1183:1380, ack 188, win 65535, length 197: HTTP
09:22:19.571237 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 1380, win 197, length 0
09:22:19.571478 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 1380:1577, ack 188, win 65535, length 197: HTTP
09:22:19.571663 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 1577, win 197, length 0
09:22:19.571810 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 1577:1774, ack 188, win 65535, length 197: HTTP
09:22:19.571898 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 1774, win 197, length 0
09:22:19.572124 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 1774:1971, ack 188, win 65535, length 197: HTTP
09:22:19.572180 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 1971, win 197, length 0
09:22:19.572333 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 1971:2168, ack 188, win 65535, length 197: HTTP
09:22:19.572412 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 2168, win 197, length 0
09:22:19.572652 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 2168:2365, ack 188, win 65535, length 197: HTTP
09:22:19.572763 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 2365, win 197, length 0
09:22:19.572875 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 2365:2562, ack 188, win 65535, length 197: HTTP
09:22:19.572958 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 2562, win 197, length 0
09:22:19.573054 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [P.], seq 2562:2705, ack 188, win 65535, length 143: HTTP
09:22:19.573137 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 2705, win 197, length 0
09:22:19.602464 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 2705:2902, ack 188, win 65535, length 197: HTTP
09:22:19.602592 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 2902, win 197, length 0
09:22:19.602830 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 2902:3099, ack 188, win 65535, length 197: HTTP
09:22:19.602928 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 3099, win 197, length 0
09:22:19.603188 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 3099:3296, ack 188, win 65535, length 197: HTTP
09:22:19.603273 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 3296, win 197, length 0
09:22:19.605576 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 3296:3493, ack 188, win 65535, length 197: HTTP
09:22:19.605694 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 3493, win 197, length 0
09:22:19.606016 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 3493:3690, ack 188, win 65535, length 197: HTTP
09:22:19.606104 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 3690, win 197, length 0
09:22:19.606281 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 3690:3887, ack 188, win 65535, length 197: HTTP
09:22:19.606369 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 3887, win 197, length 0
09:22:19.606554 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 3887:4084, ack 188, win 65535, length 197: HTTP
09:22:19.606631 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 4084, win 197, length 0
09:22:19.606781 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 4084:4281, ack 188, win 65535, length 197: HTTP
09:22:19.606862 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 4281, win 197, length 0
09:22:19.607016 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 4281:4478, ack 188, win 65535, length 197: HTTP
09:22:19.607153 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 4478, win 197, length 0
09:22:19.607316 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 4478:4675, ack 188, win 65535, length 197: HTTP
09:22:19.607412 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 4675, win 197, length 0
09:22:19.607549 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 4675:4872, ack 188, win 65535, length 197: HTTP
09:22:19.607618 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 4872, win 197, length 0
09:22:19.607755 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 4872:5069, ack 188, win 65535, length 197: HTTP
09:22:19.607826 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 5069, win 197, length 0
09:22:19.607927 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 5069:5266, ack 188, win 65535, length 197: HTTP
09:22:19.607957 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 5266, win 197, length 0
09:22:19.608032 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 5266:5463, ack 188, win 65535, length 197: HTTP
09:22:19.608071 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 5463, win 197, length 0
09:22:19.608175 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 5463:5660, ack 188, win 65535, length 197: HTTP
09:22:19.608215 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 5660, win 197, length 0
09:22:19.608316 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 5660:5857, ack 188, win 65535, length 197: HTTP
09:22:19.608347 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 5857, win 197, length 0
09:22:19.608444 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 5857:6054, ack 188, win 65535, length 197: HTTP
09:22:19.608474 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 6054, win 197, length 0
09:22:19.608567 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 6054:6251, ack 188, win 65535, length 197: HTTP
09:22:19.608597 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 6251, win 197, length 0
09:22:19.609276 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 7433:7630, ack 188, win 65535, length 197: HTTP
09:22:19.609312 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 7630, win 197, length 0
09:22:19.609400 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 7630:7827, ack 188, win 65535, length 197: HTTP
09:22:19.609436 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 7827, win 197, length 0
09:22:19.609545 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 7827:8024, ack 188, win 65535, length 197: HTTP
09:22:19.609597 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 8024, win 197, length 0
09:22:19.609671 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 8024:8221, ack 188, win 65535, length 197: HTTP
09:22:19.609723 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 8221, win 197, length 0
09:22:19.609818 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 8221:8418, ack 188, win 65535, length 197: HTTP
09:22:19.609867 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 8418, win 197, length 0
09:22:19.609987 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 8418:8615, ack 188, win 65535, length 197: HTTP
09:22:19.610038 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 8615, win 197, length 0
09:22:19.610128 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 8615:8812, ack 188, win 65535, length 197: HTTP
09:22:19.610190 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 8812, win 197, length 0
09:22:19.610305 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 8812:9009, ack 188, win 65535, length 197: HTTP
09:22:19.610337 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 9009, win 197, length 0
09:22:19.610436 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 9009:9206, ack 188, win 65535, length 197: HTTP
09:22:19.610468 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 9206, win 197, length 0
09:22:19.610558 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 9206:9403, ack 188, win 65535, length 197: HTTP
09:22:19.633567 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 9465:9662, ack 188, win 65535, length 197: HTTP
09:22:19.633616 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 9662, win 197, length 0
09:22:19.633872 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 9662:9859, ack 188, win 65535, length 197: HTTP
09:22:19.633902 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 9859, win 197, length 0
09:22:19.634149 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 9859:10056, ack 188, win 65535, length 197: HTTP
09:22:19.634177 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 10056, win 197, length 0
09:22:19.635263 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 10056:10253, ack 188, win 65535, length 197: HTTP
09:22:19.635292 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 10253, win 197, length 0
09:22:19.635521 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 10253:10450, ack 188, win 65535, length 197: HTTP
09:22:19.635549 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 10450, win 197, length 0
09:22:19.635725 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 10450:10647, ack 188, win 65535, length 197: HTTP
09:22:19.635787 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 10647, win 197, length 0
09:22:19.636000 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], seq 10647:10844, ack 188, win 65535, length 197: HTTP
09:22:19.636027 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [.], ack 10844, win 197, length 0
09:22:19.636257 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [FP.], seq 10844:10976, ack 188, win 65535, length 132: HTTP
09:22:19.636330 IP 10.0.2.15.13608 > 10.255.254.161.80: Flags [F.], seq 188, ack 10977, win 197, length 0
09:22:19.636533 IP 10.255.254.161.80 > 10.0.2.15.13608: Flags [.], ack 189, win 65535, length 0

@mikumiku2468
Copy link
Author

here is the code of test
package main

import (
"fmt"
"net/http"
"io/ioutil"
)

//var a string

func main() {
conntransport := &http.Transport{
DisableKeepAlives: true,
DisableCompression: true,
}
client := &http.Client{Transport: conntransport}
req, _ := http.NewRequest("GET", "http://10.255.254.161", nil)
req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36")

resp,err:=client.Do(req)
fmt.Println(err)
ioutil.ReadAll(resp.Body)
conntransport.CloseIdleConnections()

}

@qifengzhang007
Copy link

is this golang bug?
here is my test code


func main() {
	resp, err := http.Get("http://49.232.145.118:20171/api/v1/portal/news?newsType=10&page=1&limit=50")
	if err != nil {
		fmt.Println(err)
		return
	}

	body, err := ioutil.ReadAll(resp.Body)
	fmt.Println(string(body))
	fmt.Println(resp.StatusCode)
	if resp.StatusCode == 200 {
		fmt.Println("ok")
	}
	_ = resp.Body.Close()

}

CentOS server uses tcpdump to capture packets as follows,it shows not send FIN

19:48:38.422375 IP ..8598 > VM-0-13-centos.20171: Flags [S], seq 3292284131, win 64240, options [mss 1412,nop,wscale 8,nop,nop,sackOK], length 0
19:48:38.422510 IP VM-0-13-centos.20171 > ..8598: Flags [S.], seq 314920111, ack 3292284132, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 0
19:48:38.454949 IP ..8598 > VM-0-13-centos.20171: Flags [.], ack 1, win 512, length 0
19:48:38.455966 IP ..8598 > VM-0-13-centos.20171: Flags [P.], seq 1:180, ack 1, win 512, length 179
19:48:38.456017 IP VM-0-13-centos.20171 > ..8598: Flags [.], ack 180, win 60, length 0
19:48:38.456783 IP VM-0-13-centos.20171 > ..8598: Flags [P.], seq 1:764, ack 180, win 60, length 763
19:48:38.500583 IP ..8598 > VM-0-13-centos.20171: Flags [R.], seq 180, ack 764, win 0, length 0


@seankhliao seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Dec 26, 2022
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@golang golang locked and limited conversation to collaborators Jan 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

5 participants