HTTP, so far as I know, doesn't make any distinction between "no body" and "empty body". The choice to send the END_STREAM flag in a header frame or in an empty data frame is an implementation detail. Adding API surface to expose this detail implies that users should consider and/or take advantage of the distinction.

If we were to support this, I think there needs to be compelling evidence that this is of general value.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Some servers differentiate between the two kinds of requests. We're running a MITM proxy that needs to read which kind of request a client is making and forward the same to the target server.

@shreyas44, can you please give more detail about this? Which servers? What kinds of differentiation?

@neild's comment suggests that we probably shouldn't do this without a compelling reason.

@neild @rsc apologies for the inactivity.

HTTP, so far as I know, doesn't make any distinction between "no body" and "empty body". The choice to send the END_STREAM flag in a header frame or in an empty data frame is an implementation detail. Adding API surface to expose this detail implies that users should consider and/or take advantage of the distinction.

If we were to support this, I think there needs to be compelling evidence that this is of general value.

You're right that for the majority of the cases this doesn't matter and should remain an implementation detail. The only use case I can think of right now is ours - trying to read and replicate the behaviour of a client that does differentiate between the two (Go being one of them as mentioned previously, although I'm not sure whether that's intentional).

We could also use http.NoBody to indicate no body and an instance of the default reader that immediately returns EOF for an empty body. While this still exposes the distinction, it doesn't create any new variables/constants. This is also inline with the behaviour of http2.Transport where no body is sent if req.Body is http.NoBody and an empty body is sent if the body is any other reader that immediately returns EOF.

can you please give more detail about this? Which servers? What kinds of differentiation?

What caused issues for us was an AkamaiGhost server. When sending an empty body, we received a 403 forbidden response but when sending no body we received a successful response.

Cloudflare also seems to differentiate between the two. The crux of the linked post is that cloudflare doesn't add a Content-Length header when forwarding a h2 request with no body to a cloudflare worker but it does when there's an empty body.

I think the terms "no body" and "empty body" are adding confusion here. There is no such distinction in HTTP. All HTTP requests and responses have a payload, and the payload may have a length of zero. There aren't different types of zero-length payloads.

What you're requesting is a way to distinguish between HTTP/2 streams with a zero-length payload that send the END_STREAM flag in the last headers frame vs. the first (empty) data frame.

This is a distinguishable property of HTTP/2 streams, just as the division of the payload between data frames is, but HTTP/2 doesn't draw a semantic distinction in either case and I don't think we should be exposing either in the net/http API.

Is there any indication from Akamai or Cloudflare that this behavior is intentional, rather than just a bug in their implementations? If this is just a server-side bug in those two systems, it would be better for Go not to expose the difference and open the door to more such bugs.

As far as I know, neither Akamai nor Cloudflare have mentioned this issue anywhere. You might be right in that it's a bug and not intentional.

This is a distinguishable property of HTTP/2 streams, just as the division of the payload between data frames is, but HTTP/2 doesn't draw a semantic distinction in either case and I don't think we should be exposing either in the net/http API.

it would be better for Go not to expose the difference and open the door to more such bugs.

Fair. On the note of preventing bugs, I think x/net/http2 should set req.Body to http.NoBody in both cases instead of the current behavior where it's set to a reader that immediately returns EOF. This is a different issue however.

What you're requesting is a way to distinguish between HTTP/2 streams with a zero-length payload that send the END_STREAM flag in the last headers frame vs. the first (empty) data frame.

Do you have any suggestions on how we could do this without having to fork x/net/http2?

Do you have any suggestions on how we could do this without having to fork x/net/http2?

I'm afraid not. The net/http and x/net/http2 packages don't try to be all things to all people--there are, for example, legitimate reasons under some circumstances to care about the order of received headers, but there is no way that I know of to get this information with net/http.

Perhaps there's a case to be made for this feature (if it's something that other HTTP/2 implementations commonly expose, that would be an argument that net/http should perhaps expose it as well), but a good dividing line is that net/http should expose the RFC-defined semantic properties of HTTP requests and responses, and not expose incidental implementation details.

@neild and I are both mildly but not strongly opposed to this. We sympathize with having to interop with broken servers.

If we do this, it should be locked in with a test, have similar behavior between HTTP/1 and HTTP/2, and be documented.

I also want to see first a summary table of the current and proposed states of the server behavior for all combinations of:

• http/1 vs http/2
• content-length: 0 vs no content-length (or chunked for http/1)
• both END bits set in http2 vs two frames

It sounds like this is blocked on gathering the data table that @bradfitz asked for in the previous message, if we want to move forward.

@shreyas44, any interest in collecting the data Brad asked for in #53894 (comment) ?
That would let us move forward with this issue.
Otherwise I'll put it on hold for anyone else to collect the data.

Placed on hold.
— rsc for the proposal review group