/cc @tombergan

@smarterclayton Do you use HTTP/2 via net/http or do you use x/net/http2 directly? net/http is currently very centered around HTTP/1 and doesn't have a real notion of a "connection".

Ideas:

1. Add x/net/http2.ServeConn, with Serve and Shutdown methods. A handler could get at the request's ServeConn by extracting a value from the request's Context. This feels kind of ugly and doesn't backport nicely to net/http.

2. Define an H2-specific handler type that exposes http2.ServeConn directly. This could have other advantages, e.g., the H2-specific handler could use a concrete ResponseWriter type that has a Push method, so users could call Push without a type cast. This feels less ugly, but doesn't backport nicely to net/http.

3. Define interface { GoAway() } which is implemented by the HTTP/2 ResponseWriter. The semantics are: GoAway ends the current response and gracefully shuts down the client connection. This interface could be defined in net/http. We are starting to collect interfaces like this (also see http.Pusher and http.CloseNotifier), which feels ugly.

4. Similar to the above, but instead the interface is something like interface { SetCloseConnectionAfterResponse() }. The semantics are: the method must be called before WriteHeader. In HTTP/1, the response will have Connection: close. In HTTP/2, a GOAWAY will be sent after the response. This works identically for H1 and H2, which is nice, but I can't think of a non-ugly method name, which is not nice.

5. Reuse Connection: close in HTTP/2. I don't like this idea. This could result in surprising behavior, for example, for people who write H2 -> H1 proxies and forget to remove hop-by-hop headers when generating the proxied response.

@bradfitz, any thoughts?

Somewhat relatedly, the godoc for net/http.Handler says the following, which is actually not true for HTTP/2. (In H2, we send a RST_STREAM on panic.)

If ServeHTTP panics, the server ... recovers the panic, logs a stack trace to the server error log, and hangs up the connection.

What do you actually want to do?

(1) hard-close the TCP connection and break all other streams currently in-flight?
(2) just send a GOAWAY, gracefully finishing the current active requests?
(3) send a REFUSED_STREAM, so the client retries it elsewhere?

I don't want to overload Connection: close for sure.

Another possibility is magic panic values, since net/http has historically (for better or worse) defined that panic is caught by the server and mapped into a server error. We can say that if the panic value implements a certain interface (say, interface { SendGoAway() } or interface { SendRefusedStream() } etc), then we do that. Then it also doesn't matter type-wise if the user uses the bundled x/net/http2 in std but uses the panic values from x/net/http2.

But let's figure out the goals before we design.

Kubernetes may be a bit unusual in this regard in that it has large numbers of very long lived connections from large numbers of stable clients (infrastructure) that process large numbers of individual requests. To avoid wasting CPU on TLS setup, we work very hard to avoid closing connections unless necessary. However, occasionally a backend server is overloaded and wishes to shed load. To do that, we send 429. However, with lots of stable clients (including browsers), the 429 doesn't actually shed load from that particular backend server, because clients continue to use that connection as long as it is open. This leads to rolling updates sloshing large chunks of load around, without balancing them (because the moment the server goes down, the clients reconnect because of the high rate of requests).

For this use case, for HTTP1, closing the TCP connection works well because intervening HTTP and TCP proxies handle that very well. Since that's one request in flight at a time, closing is acceptable because it does not impact other clients.

For HTTP2, given the lack of advanced HTTP2 support in most of the proxies in common use, closing the TCP connection is the only mechanism that would move load around. We would prefer it be graceful. So that sounds like 2 above (we want the client to GOAWAY and to hit the load balancer again). But if REFUSED_STREAM would signal a reasonable HTTP2 client to create a new connection, that would be desirable.

(3) send a REFUSED_STREAM, so the client retries it elsewhere?
But if REFUSED_STREAM would signal a reasonable HTTP2 client to create a new connection, that would be desirable.

Servers cannot rely on REFUSED_STREAM to mean "try again on a different connection". That is not what the HTTP/2 spec says and it's not what is implemented by Chrome or Firefox. See code links here.

I think GOAWAY is the right mechanism for this.

/cc @rs

A quick solution is to have an exported version http2errClientConnClosed,
so we can just compare and ignore it.

@kelwang, I think you're confusing this issue with a different one.

It turns out that Go's HTTP/2 server was improperly allowing Handlers to send a "Connection: close" response header, which is a protocol violation.

I'll kill two birds with one stone and while fixing that, also fix this bug and make the "Connection: close" also mean to send a GOAWAY, like we do for HTTP/1.x.

Change https://golang.org/cl/121415 mentions this issue: http2: make Server send GOAWAY if Handler sets "Connection: close" header

Change https://golang.org/cl/121695 mentions this issue: net/http: update bundled http2

Guys what's the latest update on this? We just ran into this, & I spent the whole day in investigating why some clients are not closing connection, only to figure out HTTP2 clients are not receiving it at all.
What do I do server side to force HTTP2 clients to close the connection?

I'm using ASP .NET server side btw.

Send response header Connection: close.

That doesn't work right, we're already sending it. Http/2 clients do not receive this header, only Http/1 clients do. I thought that's what this whole thread is about, to cfigure out a way to close Http/2 connections.

Maybe a stupid question, but you are using the latest Go master version right ? The fix is obviously not available in 1.10, or even the 1.11beta1 I believe.

If you still see this issue in latest master, please feel free to raise a new issue with all the necessary details. Thank you.

@AjinkyaKher, HTTP/2 clients are not supposed to see a Connection header. See golang/net@97aa3a5 for details. You need to be using a recent (unreleased) Go version to see this. Or wait a couple hours.