net/http: missing HTTP KeepAlive timeout configuration #13998
Comments
ianlancetaylor
changed the title
|
I don't understand how this could happen. Go doesn't wait to do a read on the socket when send your next request (e.g. 5.5 minutes later, past the firewall's deadline). Instead, Go is always in a read, waiting for it to hang up on you. If it hangs up in the meantime (say, at 5 minutes, per your example), that connection is removed from the pool and not used later in 5.5 minutes. Is this a hypothetical bug report or something you're actually seeing? If you're actually seeing it, please provide code and a packet capture. |
|
This is an actual bug that I see. It happens because the connection is terminated by a component in between and a FIN package is never sent back to Go. If you don't have TCP KeepAlive enabled, Go will have no way of knowing that the connection is down. It will find the first time it tries to send a new HTTP request. (I have replaced the IP addresses with source and target) You can see that the last 2 records happen around 7 minutes and 5 seconds later and comprise of a If I were to enable TCP Keep Alive, this would not be reproducible as there would be multiple (These are examples from today, that's why there is a timestamp difference) Still, it requires a conscious and educated decision when creating the |
|
We also have such a timeout set by default, if you use any of the default methods to fetch URLs (http.Get, http.Post, http.DefaultClient, etc). That default is here: var DefaultTransport RoundTripper = &Transport{
Proxy: ProxyFromEnvironment,
Dial: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).Dial,
TLSHandshakeTimeout: 10 * time.Second,
}It's true that if you go lower-level and wire stuff up yourself, you get what you ask for. I suppose a question is whether we should have a DefaultDialer too, and document that custom Transports with a nil Dial func end up using DefaultDial. (But another question is why your network component drops connections without sending a FIN, but I understand that things do weird things.) Related: at least in Go 1.6, idempotent requests are retried on a new connection, which mitigates this partially. |
|
@bradfitz This are some very good remarks and ideas. Having a |
|
Fixed by https://golang.org/cl/22670? |
|
Yup, thanks. |
|
Thanks! |
Hi,
Currently it seems that it's only possible to enable or disable
HTTP KeepAlive. The following code shows how to disable it:There does not seem to be configuration which allows you to specify a KeepAlive timeout, which would abandon idle connections that are inactive for longer than the timeout.
This is important as clients operating behind a Firewall / Proxy or targeting a server running behind a Firewall / Load Balancer might get their TCP connection closed after some period (e.g. 5 minutes) without getting informed. When this happens, Go fails the next HTTP call with
read: connection reset by peer.I faced such a scenario and checked Ruby to see how it would handle. It has multiple protections in place so that it does not error with
read: connection reset by peer.It is important to note that I am asking for an HTTP KeepAlive Timeout configuration. There is already a TCP KeepAlive Timeout configuration available in Go, which works just fine, but has a different purpose - to health-check a TCP connection by pushing regular ACK packages.
The text was updated successfully, but these errors were encountered: