-
Notifications
You must be signed in to change notification settings - Fork 18k
x/net/http2: client sometimes sends extra zero-len DATA frame after request DATA #32254
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
It's doing whatever the underlying io.Reader does. If the underlying reader returns (non-zero, io.EOF), then it sends one 1 DATA frame. (We tried to change the strings.Reader and bytes.Reader implementations several times in the past to do eager EOF returns but it broke too many tests to be worth it.) But I suppose we could know better in the case where the request body's content-length was predeclared (as it is when you use http.NewRequest on a few known types like strings.Reader). Once we've read enough bytes then we could assume that no bytes will remain on that reader, but that's also where we currently double-check users' Reader implementations and abort the http2 stream if their declared request content-length doesn't match reality. But I suppose we could trust standard library types and only sanity check unknown types. But that's a lot of special casing for minimal benefit. |
The spec (https://http2.github.io/http2-spec/#malformed) says:
So we can do the following:
Another way is that we can wrap io.Reader into a new io.Reader with content-length constraint. This can be done as a separate package outside x/net, which will have minimal side-effects. @bradfitz Which way looks good to you? I can do the implementation. |
I see that io.LimitReader will do the trick. |
I reopened the issue because io.LimitedReader does not do the trick, as it also returns EOF lazily. |
But then how/when do we report an error if the user's Request.Body Reader returns more bytes than the declared ContentLength? |
@bradfitz A simplified solution would be to drop the extra bytes silently without producing any actual error except a debug log. A complete solution would be the following:
Does these sounds OK? |
That doesn't sound okay to me. I'm not even sure how (1) would work, and even if it did, it'd be silently sending truncated data to the peer, rather than aborting the stream which is meant to prevent them fro working with corrupted data. |
Fair enough. |
Change https://golang.org/cl/181157 mentions this issue: |
@bradfitz Hi Brad, I thought it may be easier to just talk about the code, so I filed https://golang.org/cl/181157 What I did is also simpler than discussed before. It fixes the issue, while still allows sending any body without interruption or truncation, even if the actual length does not match specified content-length. This is exactly the behavior before. I thought it is the receiving peer's responsibility to handle the malformed request body rather than the sending peer. |
Check EOF eagerly on the request body when its content-length is specified and it is expected to end. Thus, the data frame containing the last chunk of data of the body will be marked with END_STREAM eagerly. However, if the specified content-length is incorrect, the whole request body will still be sent, regardless of its actual length. This will render the request malformed according to the spec. We believe it is the receiving peer's responsibility to handle the error. Fix golang/go#32254 Change-Id: Id24c043c7cc3a41421dfd099a139f1b1e08056b9
@dmitshur Since Brad is on leave, can you please review https://golang.org/cl/181157 ? Thanks! |
ping |
Change https://golang.org/cl/200102 mentions this issue: |
Updates x/net/http2 to git rev d66e71096ffb9f08f36d9aefcae80ce319de6d68 http2: end stream eagerly after sending the request body https://golang.org/cl/181157 (fixes #32254) all: fix typos https://golang.org/cl/193799 http2: fix memory leak in random write scheduler https://golang.org/cl/198462 (fixes #33812) http2: do not sniff body if Content-Encoding is set https://golang.org/cl/199841 (updates #31753) Also unskips tests from CL 199799. Change-Id: I241c0b1cd18cad5041485be92809137a973e33bd Reviewed-on: https://go-review.googlesource.com/c/go/+/200102 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
Check EOF eagerly on the request body when its content-length is specified and it is expected to end. Thus, the data frame containing the last chunk of data of the body will be marked with END_STREAM eagerly. In case the request body is larger than the specified content-length, the request will be aborted and returned with an error. Fixes golang/go#32254 Change-Id: Id24c043c7cc3a41421dfd099a139f1b1e08056b9 Reviewed-on: https://go-review.googlesource.com/c/net/+/181157 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes. I have tried with the latest master branch of golang.org/x/net/http2 (commit f3200d17e092c607f615320ecaad13d87ad9a2b3).
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
Code: https://play.golang.org/p/SOgAnCu0ybi
Run with GODBUG=http2debug=2
What did you expect to see?
Client should write a HEADERS frame and a single DATA frame. The payload "hello, world!" should be in the DATA frame, with END_STREAM set.
What did you see instead?
There are 2 DATA frames. First one with the actual payload, second one with END_STREAM flag.
Strictly speaking, this does not violate the specs. However, there are several reasons that we should save the extra frame:
The text was updated successfully, but these errors were encountered: