-
Notifications
You must be signed in to change notification settings - Fork 18k
compress/flate: crash called from multiple goroutines #2779
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Labels
Milestone
Comments
For good measure, in case it's significant that the call to Read happens from another goroutine: func (t TimeoutReader) Read(buf []byte) (int, error) { ch := make(chan timeoutReadDone) go func() { n, err := t.Reader.Read(buf) ch <- timeoutReadDone{n, err} }() select { case done := <-ch: return done.n, done.err case <-t.Timeout: break } return 0, ErrReadTimeout } |
Do you ever keep reading from a timeout reader after one timeout has occurred? If so, you'd have multiple calls to t.Reader.Read going at once, which is typically not allowed by most readers, and it could explain the crash. Can you reproduce the crash without the timeout reader? The gzip decompressor is completely deterministic, so if not, that's a strong signal that the timeout reader is causing what I was describing. Russ |
I'll disable it and see if I can reproduce it. This is how I'm using the timeoutReader: (Including this just in case the goto could have some effect on the res.Body reader) RETRY: res, err := c.Get(u) if err != nil { if strings.HasSuffix(err.Error(), "Temporary failure in name resolution") { <-time.After(10 * time.Second) goto RETRY } log.Println("Error opening", u, "-", err) } else { r = Process(res.Body) } ... func Process(io.Reader) { tr := timeoutReader(res.Body, time.After(10*time.Second)) z := html.NewTokenizer(tr) L: for { tt := z.Next() switch tt { case html.StartTagToken... ... case html.ErrorToken { break L } } } It's discarded after that. Perhaps a bug with Tokenizer? Best as I can tell, although its error handling is unconventional, readByte's change of z.err is never ignored. I'll let you know. Patrick |
It is okay for something processing an io.Reader to call Read again after getting an error. The timeoutReader you've shown does not handle this scenario: it will end up with two different Read calls going on in two different goroutines. I think you want something like this instead. during init, t.canRead = make(chan bool, 1); t.canRead <- true func (t *TimeoutReader) Read(buf []byte) (int, error) { // Wait for permission to start a new Read. select { case <-t.canRead: // ok case <-t.Timeout: return 0, ErrReadTimeout } // Start a new Read. ch := make(chan timeoutReadDone) go func() { n, err := t.Reader.Read(buf) ch <- timeoutReadDone{n, err} t.canRead <- true }() // Wait for Read. select { case done := <-ch: return done.n, done.err case <-t.Timeout: break } return 0, ErrReadTimeout } If you serialize the Read calls that way, do you still see the problem? Labels changed: added priority-go1, removed priority-triage. Status changed to WaitingForReply. |
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The text was updated successfully, but these errors were encountered: