Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compress/flate: crash called from multiple goroutines #2779

Closed
patrickmn opened this issue Jan 25, 2012 · 6 comments
Closed

compress/flate: crash called from multiple goroutines #2779

patrickmn opened this issue Jan 25, 2012 · 6 comments
Milestone

Comments

@patrickmn
Copy link

What steps will reproduce the problem? Unknown

Which compiler are you using (5g, 6g, 8g, gccgo)? 8g

Which operating system are you using? Ubuntu 11.10

Which revision are you using? weekly/weekly.2012-01-20

Please provide any additional information below.

h.codes[v-h.base[n]] fails, presumably while processing bad input. Unfortunately, I've
yet to reproduce.

goroutine 188467 [running]:
compress/flate.(*decompressor).huffSym(0x1e5e1000, 0x1e5e1020, 0x9d9dfdf4, 0x9d9dfdbc)
    /home/ubuntu/apps/go/src/pkg/compress/flate/inflate.go:659 +0x1a9
compress/flate.(*decompressor).huffmanBlock(0x1e5e1000, 0xb767c090)
    /home/ubuntu/apps/go/src/pkg/compress/flate/inflate.go:400 +0x39
compress/flate.(*decompressor).copyHuff(0x1e5e1000, 0x1)
    /home/ubuntu/apps/go/src/pkg/compress/flate/inflate.go:553 +0x15c
compress/flate.(*decompressor).Read(0x1e5e1000, 0x24c5e008, 0xff8, 0xff8, 0x4, ...)
    /home/ubuntu/apps/go/src/pkg/compress/flate/inflate.go:284 +0x117
compress/gzip.(*Decompressor).Read(0x1cecda00, 0x24c5e008, 0xff8, 0xff8, 0x60, ...)
    /home/ubuntu/apps/go/src/pkg/compress/gzip/gunzip.go:208 +0xd1
net/http.(*discardOnCloseReadCloser).Read(0x1d543618, 0x24c5e008, 0xff8, 0xff8,
0x815a728, ...)
    /home/ubuntu/apps/go/src/pkg/net/http/chunked.go:0 +0x4f
net/http.(*readFirstCloseBoth).Read(0x1cd35c10, 0x24c5e008, 0xff8, 0xff8, 0x1, ...)
    /home/ubuntu/apps/go/src/pkg/net/http/chunked.go:0 +0x4f
net/http.(*bodyEOFSignal).Read(0x1cd35c20, 0x24c5e008, 0xff8, 0xff8, 0x2646b300, ...)
    /home/ubuntu/apps/go/src/pkg/net/http/transport.go:686 +0x4f
main._func_003(0x25ae2ba0, 0x25ae2bb0, 0x26038730, 0x18dddf10)
    /home/ubuntu/bug-crawler/main.go:517 +0x45
created by main.TimeoutReader.Read
    /home/ubuntu/bug-crawler/main.go:519 +0xb4

TimeoutReader is an io.Reader which checks (select) a timeout chan after each Read.
@patrickmn
Copy link
Author

Comment 1:

For good measure, in case it's significant that the call to Read happens from another
goroutine:
func (t TimeoutReader) Read(buf []byte) (int, error) {
        ch := make(chan timeoutReadDone)
        go func() {
                n, err := t.Reader.Read(buf)
                ch <- timeoutReadDone{n, err}
        }()
        select {
        case done := <-ch:
                return done.n, done.err
        case <-t.Timeout:
        break
        }
        return 0, ErrReadTimeout
}

@rsc
Copy link
Contributor

rsc commented Jan 25, 2012

Comment 2:

Do you ever keep reading from a timeout reader
after one timeout has occurred?  If so, you'd have
multiple calls to t.Reader.Read going at once,
which is typically not allowed by most readers,
and it could explain the crash.
Can you reproduce the crash without the timeout
reader?  The gzip decompressor is completely
deterministic, so if not, that's a strong signal that
the timeout reader is causing what I was describing.
Russ

@patrickmn
Copy link
Author

Comment 3:

I'll disable it and see if I can reproduce it.
This is how I'm using the timeoutReader:
(Including this just in case the goto could have some effect on the res.Body reader)
RETRY:
        res, err := c.Get(u)
        if err != nil {
                if strings.HasSuffix(err.Error(), "Temporary failure in name resolution") {
                        <-time.After(10 * time.Second)
                        goto RETRY
                }
                log.Println("Error opening", u, "-", err)
        } else {
                r = Process(res.Body)
        }
...
func Process(io.Reader) {
        tr := timeoutReader(res.Body, time.After(10*time.Second))
        z := html.NewTokenizer(tr)
L:
        for {
            tt := z.Next()
            switch tt {
               case html.StartTagToken...
               ...
               case html.ErrorToken {
                   break L
               }
            }
        }
It's discarded after that.
Perhaps a bug with Tokenizer? Best as I can tell, although its error handling is
unconventional, readByte's change of z.err is never ignored. 
I'll let you know.
Patrick

@rsc
Copy link
Contributor

rsc commented Jan 25, 2012

Comment 4:

It is okay for something processing an io.Reader to call Read again after getting an
error.  The timeoutReader you've shown does not handle this scenario: it will end up
with two different Read calls going on in two different goroutines.  I think you want
something like this instead.
during init, t.canRead = make(chan bool, 1); t.canRead <- true
func (t *TimeoutReader) Read(buf []byte) (int, error) {
        // Wait for permission to start a new Read.
        select {
        case <-t.canRead:
            // ok
        case <-t.Timeout:
            return 0, ErrReadTimeout
        }
        // Start a new Read.
        ch := make(chan timeoutReadDone)
        go func() {
                n, err := t.Reader.Read(buf)
                ch <- timeoutReadDone{n, err}
                t.canRead <- true
        }()
        // Wait for Read.
        select {
        case done := <-ch:
                return done.n, done.err
        case <-t.Timeout:
        break
        }
        return 0, ErrReadTimeout
}
If you serialize the Read calls that way, do you still see the problem?

Labels changed: added priority-go1, removed priority-triage.

Status changed to WaitingForReply.

@patrickmn
Copy link
Author

Comment 5:

Hi Russ,
I haven't been able to reproduce it (with any of the timeoutReaders), but what you're
saying makes complete sense. Please feel free to close this.
Thanks for taking the time to elaborate.

@rsc
Copy link
Contributor

rsc commented Jan 26, 2012

Comment 6:

Status changed to Retracted.

@rsc rsc added this to the Go1 milestone Apr 10, 2015
@rsc rsc removed the priority-go1 label Apr 10, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants