Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

flate: index out of range when compressing at level 1 #2508

Closed
remyoudompheng opened this issue Nov 29, 2011 · 9 comments
Closed

flate: index out of range when compressing at level 1 #2508

remyoudompheng opened this issue Nov 29, 2011 · 9 comments
Labels
FrozenDueToAge
Milestone
Go1

Comments

@remyoudompheng
Copy link
Contributor 

What steps will reproduce the problem?

1. Compile and run this:

package main

import (
    "compress/flate"
    "io/ioutil"
)

func main() {
    w := flate.NewWriter(ioutil.Discard, 1)
    defer w.Close()
    n := int64(5 << 25)
    buf := make([]byte, 1024)
    for i := 0; i < int(n>>10); i++ {
        w.Write(buf)
    }
}

What is the expected output?

No error.

What do you see instead?

panic: runtime error: index out of range

goroutine 1 [running]:
compress/flate.(*compressor).deflate(0xf840021000, 0xf840003800)
    /opt/remy/go/src/pkg/compress/flate/deflate.go:322 +0x60e
compress/flate.(*compressor).close(0xf840021000, 0x7f83074a2f40, 0xf840020840, 0x400ced)
    /opt/remy/go/src/pkg/compress/flate/deflate.go:413 +0x2d
compress/flate.(*Writer).Close(0xf840021000, 0x200000000000, 0x1d000000000001,
0xf800000400)
    /opt/remy/go/src/pkg/compress/flate/deflate.go:491 +0x25
main.main()
    /tmp/test/zip.go:13 +0xed


Which compiler are you using (5g, 6g, 8g, gccgo)?

6g

Which operating system are you using?

Linux, amd64.

Which revision are you using?  (hg identify)

1713e0b34111+ tip (rev 10607)
@remyoudompheng
Copy link
Contributor Author

Comment 1:

Simpler test case :
package main
import (
    "compress/flate"
    "io/ioutil"
)
func main() {
    w := flate.NewWriter(ioutil.Discard, 1)
    buf := make([]byte, 1024)
    for i := 0; i < 131072; i++ {
        w.Write(buf)
    }
    w.Close()
}
The problem disappears for 131071 instead of 131072.

@rsc
Copy link
Contributor

rsc commented Dec 7, 2011

Comment 2:

confirmed

Status changed to Accepted.

@krasin
Copy link

krasin commented Dec 7, 2011

Comment 3:

This looks to be off-by-1 error. Just before the crash:
d.index: 65536, d.length: 193, len(d.window): 65536
I will continue to look into this.

@krasin
Copy link

krasin commented Dec 7, 2011

Comment 4:

Now, the situation is pretty clear. If findMatch uses the whole lookahead and lookahead
> d.fastSkipHashing, then deflate may be unable to calculate the next hash, because it
might the data outside of the window.
I'm thinking about the appropriate fix for that. Will update this thread soon.

@krasin
Copy link

krasin commented Dec 8, 2011

Comment 5:

The fix is here: http://golang.org/cl/5449115/
The CL is not ready to be committed, because I want to add the test as well, but it's
ready for the review.

@rsc
Copy link
Contributor

rsc commented Dec 9, 2011

Comment 6:

Labels changed: added priority-later, removed priority-medium.

@robpike
Copy link
Contributor

robpike commented Dec 12, 2011

Comment 7:

Status changed to Started.

@rsc
Copy link
Contributor

rsc commented Dec 12, 2011

Comment 8:

Labels changed: added priority-go1.

@rsc
Copy link
Contributor

rsc commented Dec 12, 2011

Comment 9:

This issue was closed by revision b1ae728.

Status changed to Fixed.

@rsc rsc added this to the Go1 milestone Apr 10, 2015
@rsc rsc removed the priority-go1 label Apr 10, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1
Development

No branches or pull requests
5 participants
@krasin @rsc @remyoudompheng @robpike @gopherbot