Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proposal: net/http: Allow configurable max connection age #43905

Open
dmarkhas opened this issue Jan 25, 2021 · 6 comments
Open

proposal: net/http: Allow configurable max connection age #43905

dmarkhas opened this issue Jan 25, 2021 · 6 comments
Labels
Proposal
Projects
Proposals (old)
Milestone
Proposal

Comments

@dmarkhas
Copy link

Currently, the http.Transport implementation will reuse an HTTP connection if keepalives are enabled and the connection has not been in an idle state for too long.
This means that for high throughput connections that are always active and rarely/never exceed the IdleConnTimeout limit, the connection will keep getting reused (as long as it is not broken).

This poses a problem in terms of service discovery and client-side load balancing - since address resolution only happens in the dialer's DialContext, this leaves no chance for the client to re-resolve the target address since the same connection is constantly reused without ever "re-dialing".

I propose to add a MaxConnectionAge flag to http.Transport that would limit the lifetime of a persistent connection, and would not reuse it beyond this lifetime.

This would be in line with the behavior of go-grpc which also enforces a MaxConnectionAge limit.
@seankhliao seankhliao changed the title net/http: Allow configurable max connection age proposal: net/http: Allow configurable max connection age Jan 25, 2021
@gopherbot gopherbot added this to the Proposal milestone Jan 25, 2021
@networkimprov
Copy link

cc @bradfitz @neild

@RobGraham
Copy link

Similar to #23427 which offers a work around (hack). However re-resolving DNS and reusing connection issues have been apparent for many years.

@szuecs
Copy link

szuecs commented Feb 15, 2021

Maybe the TTL based DNS resolution could be a solution #23427 (comment).
#23427 (comment) summarized 3 possible solutions.

I am pretty sure that in practice you never max out the number of connections, so you always have idle conns that will be closed in a "hack" as I showed in #23427. We run this in a high traffic scenario since some years without an issue that we could observe (does not mean that it does not exist in practice).

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Feb 17, 2021
@raidancampbell
Copy link

I have a different issue that would be resolved by the same configuration. For reasons outside my control, I have a firewall that kills TCP connections after 15 minutes regardless of their activity. As mentioned above MaxConnectionAge solves this server-side in gRPC.

Having a similar configuration would be an easy client-side fix for me in HTTP, but I'll readily admit this problem is not likely common enough to warrant a change to net/http.

@jpcope
Copy link

jpcope commented Feb 8, 2022

Note, I believe I have a working implementation proposal at #46714 and it may be reusable for addressing #23427

@jpcope
Copy link

jpcope commented Sep 13, 2022

Also, sorry for the potentially dupe proposal. I believe the maintainers were not interested in reviewing my PR without a proposal of my own for my specific implementation.

#54429

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Proposal
Projects
Proposals
Status: Incoming
Proposals (old)
Incoming
Milestone
Proposal
Development

No branches or pull requests
8 participants
@szuecs @networkimprov @RobGraham @raidancampbell @gopherbot @jpcope @seankhliao @dmarkhas