As a workaround, it should be possible to call syscall.Setsockoptint on the FD of the socket. Example here for different socket options: https://stackoverflow.com/questions/40544096/how-to-set-socket-option-ip-tos-for-http-client-in-go-language#40549614

CC @neild

Hello, any updates, discussions, ideas?

The argument for why there is a default keep-alive value is #23459.

It is of course possible for a program to change the default, as I think you know.

Are you suggesting that we change the default behavior on android and ios? If so, what should we change it to?

The original idea was to have about 3 minutes for dead peer detection, but the current implementation does that suboptimal.

RIght now it is achieved by sending 10 keep-alive packets every 15 seconds (on Linux), after which, if the peer did not respond to any of them, the connection is considered broken after 2 minutes 30 seconds (15 + 15*9).

TCP_KEEPIDLE=15
TCP_KEEPINTVL=15
TCP_KEEPCNT=9

However, the same could be achieved without spending too much battery life, by configuring bigger initial timeout and smaller number of packets TCP_KEEPCNT. Like this:

TCP_KEEPIDLE=180
TCP_KEEPINTVL=15
TCP_KEEPCNT=2

This configuration begins probing the client only after 3 minutes (180 seconds) of inactivity on the socket. After 2 keep-alive packets with 15 seconds interval, or 3 minutes 30 seconds, the connection would be considered broken.

Are you suggesting that we change the default behavior on android and ios?

This should be changed globally, because changing it only on Android/iOS would still affect the devices connecting to the servers with low timeouts.

Please also take a look at the arguments here: #23459 (comment)
It states the problems I'm describing in this ticket, and it's from 2018.

I also believe that 3 minute timeout is still low, consider setting it to at least 5 minutes.

Note that I think there are some portability concerns here. For example, as far as I know OpenBSD does not permit setting these values individually for each socket.

Regarding "why 5 minutes":

The mean time between two metro stations in Saint Petersburg, Russia is about 2 minutes. Cellular connectivity of my operator is available only on the stations, but not during the trip between stations. Since waiting time on the station is about 20-30 seconds only, my cellphone does not manage to find the network and connect to it all the time, and additional 2 minutes are needed to get to another station and connect there.
Despite this being a very selfish calculation method, I think it's a pretty realistic scenario of cellular connectivity loss on a global scale. It usually takes 2-3 minutes to move between connectivity spots on a train, metro, under the bridge, at least in Europe. It usually takes 2-3 minutes to go make a tea/coffee from the workplace to the kitchen in the office, and return back. I hope you got the idea.

Nothing happened for 1.19. Moving to 1.20.

Personally i think the main problem here is not the count but the problem that we are not able to maintain keep alive interval and idle timeout seperatly.

We also have a similar problem with tcp keep alive mechanism in golang. It is practically useless for us because of this issue since 2014 have a look here please too: #8328

Also my comment here:
#8328 (comment)

Being able to have a high idle time would prevent flodding the network draining bateries and having at the same time a low interval would help to dectect failed connections fast. That would be enough for us. Ofcourse haveing also possibility to control keep alive count would be good too but maybe this would not even necessary because you could adjust the interval according to your needs.

gRPC-Go is running into an issue with how TCP keepalives are configured by Go.

gRPC supports HTTP/2 level keepalives and these are configurable by the user by specifying a keepalive.Time and keepalive.Timeout value.

• keepalive.Time is analogous to tcp_keepalive_time, and
• keepalive.Timeout can be considered analogous to tcp_keepalive_intvl * tcp_keepalive_probes

gRPC also sets the TCP_USER_TIMEOUT to the value of keepalive.Timeout or to a default value of 20s if this field is unspecified by the user.

With the defaults set by Go, after a period of inactivity of 15s, a TCP keepalive probe is sent (irrespective of the value configured for keepalive.Time). And if TCP_USER_TIMEOUT is set to the default 20s, and the keepalive probe is not ACKed within 20s, the connection gets closed.

This is not the case with other language implementations of gRPC since they get the default values configured by the OS.

What is your recommendation to gRPC-Go for working around this behavior of Go?

cc: @dfawley @ejona86

#62254 proposes changing the net package APIs which set the keep-alive period to set only TCP_KEEPIDLE, not TCP_KEEPINTVL. It also proposes a new set of APIs which would permit setting TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT independently.

Change https://go.dev/cl/542275 mentions this issue: net: add KeepAliveConfig and implement SetKeepAliveConfig