net/http: ChunkedReader blocks Read until the next chunk arrives #17355
Comments
|
Hello there @yauhen-l, I think the issue here is that you io.Pipe() is not being used properly, you forgot to close the pipe writer in your write goroutine so that means the buffer will instead get full before the program is done since you never explicitly closed it and also the io.Pipe() docs mention that IMO there is no bug here, just a forgotten |
|
@odeke-em, not quite. More minimal example: https://play.golang.org/p/RuDNKvAQhT Which says: The report is that an See func (cr *chunkedReader) Read(b []uint8) (n int, err error) {
for cr.err == nil {
if cr.n == 0 {
if n > 0 && !cr.chunkHeaderAvailable() {
// We've read enough. Don't potentially block
// reading a new chunk header.
break
}
cr.beginChunk()
continue
}
if len(b) == 0 {
break
}
rbuf := b
if uint64(len(rbuf)) > cr.n {
rbuf = rbuf[:cr.n]
}
var n0 int
n0, cr.err = cr.r.Read(rbuf)
n += n0
b = b[n0:]
cr.n -= uint64(n0)
// If we're at the end of a chunk, read the next two
// bytes to verify they are "\r\n".
if cr.n == 0 && cr.err == nil {
if _, cr.err = io.ReadFull(cr.r, cr.buf[:2]); cr.err == nil {
if cr.buf[0] != '\r' || cr.buf[1] != '\n' {
cr.err = errors.New("malformed chunked encoding")
}
}
}
}
return n, cr.err
}So we it has data to return, but it's blocking further to make sure the next two bytes are valid. It could instead delay that check if it kept more state. |
|
@bradfitz awesome, thank you for the analysis and correction. Ah I see, my remedy of pw.Close() then just masked the problem and completed the writes. |
quentinmit
changed the title
quentinmit
added
the
NeedsFix
|
CL https://golang.org/cl/31329 mentions this issue. |
What version of Go are you using (
go version)?go version go1.7.1 linux/amd64What operating system and processor architecture are you using (
go env)?What did you do?
I've noticed that sometimes client(iOS, Android application) connected to my server sends chunks of
chunkedHTTP Transfer-Encoding in manner when end of current chunk is received in the next packet. E.g., client sends chunks every second:hello1\n,hello2\n, etc. Here is encoded result, where each line is sent with 1 second delay:Simulation on playground:
https://play.golang.org/p/Y9I7vYoGbx
What did you expect to see?
I expect to read chunk when it is ready in spite of how it is received: in one packet or in two.
What did you see instead?
I cannot read chunk until:
a) client closes connection
b) buffer of
ChunkedReaderis fullc) client sends line which ends with
\r\nThe text was updated successfully, but these errors were encountered: