You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I expect to see correct limiter behavior on the input request body stream regardless of:
the size body (lower or greater than 4kB, size of http chunk?)
send the response back or not (the io.Discard)
What did you see instead?
With body_3kB.jsonl.gz, both variant of program works as expected. You can change the max size for limiter http.MaxBytesReader. It limits correctly according max defined size. The behaviors of limiter and http.Server are correct.
With body_4kB.jsonl.gz, the 1st program that acctually sends response back returns unexpected error (even if body is smaller than 5kB). Furthermore, json decoder always returns an error whatever the max size of http.MaxBytesReader (100kB, 1024kB ...).
The behavior is same with io.NopCloser(io.LimitReader()) instead of http.MaxBytesReader()
The text was updated successfully, but these errors were encountered:
jerome-laforge
changed the title
[http] unexpected behavior http.Server with http.MaxBytesReader and streaming response
http: unexpected behavior http.Server with http.MaxBytesReader and streaming response
Jun 19, 2021
jerome-laforge
changed the title
http: unexpected behavior http.Server with http.MaxBytesReader and streaming response
http: unexpected behavior of http.Server with http.MaxBytesReader and streaming response
Jun 19, 2021
As workaround, I buffer the response until I read completely the request.
In order to limit the GC and consumption of resource, I recycle the the bytes.Buffer with sync.Pool that is in charge of create bytes.Buffer with initial capacity.
initially on https://groups.google.com/g/golang-nuts/c/4RDmrEYJlYE
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env
)?What did you do?
I have program with two variants.
The first program (https://play.golang.org/p/JRyrfOh9UZC):
The second program (https://play.golang.org/p/WVDK4ivA-Yj) does almost the same things except it discards the response via io.Discard.
The request bodies can be find just below (please ungzip before) and they are sent with simple curl just below:
curl -X POST 'http://127.0.0.1:8081' --data-binary '@body_3kB.jsonl'
curl -X POST 'http://127.0.0.1:8081' --data-binary '@body_4kB.jsonl'
`
What did you expect to see?
I expect to see correct limiter behavior on the input request body stream regardless of:
What did you see instead?
With body_3kB.jsonl.gz, both variant of program works as expected. You can change the max size for limiter
http.MaxBytesReader
. It limits correctly according max defined size. The behaviors of limiter and http.Server are correct.With body_4kB.jsonl.gz, the 2nd program with io.Discard works correctly.
With body_4kB.jsonl.gz, the 1st program that acctually sends response back returns unexpected error (even if body is smaller than 5kB). Furthermore, json decoder always returns an error whatever the max size of http.MaxBytesReader (100kB, 1024kB ...).
The behavior is same with
io.NopCloser(io.LimitReader())
instead ofhttp.MaxBytesReader()
The text was updated successfully, but these errors were encountered: