Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compress/flate: “corrupt input before offset” decompressing stream produced by zlib #25846

Closed
zeslava opened this issue Jun 12, 2018 · 4 comments
Closed

compress/flate: “corrupt input before offset” decompressing stream produced by zlib #25846

zeslava opened this issue Jun 12, 2018 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.11

Comments

@zeslava
Copy link

zeslava commented Jun 12, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"

What did you do?

I try to decompress stream from zlib library(deflate compression rfc1951) with Go package compress/flate. I've got error: "corrupt input before offset".
If decompress same stream with zlib library, then it's ok.

Example code: https://play.golang.org/p/sPp3uqGuHzI

What did you expect to see?

In example, byte array contains bytes from zlib deflate function output. An expected decompressed data is: "Hello Hello Hello Hello Hello Hello!"

What did you see instead?

flate: corrupt input before offset
@bcmills bcmills changed the title decompressing deflated stream by zlib library with compress/flate package compress/flate: “corrupt input before offset” decompressing stream produced by zlib Jun 12, 2018
@bcmills
Copy link
Contributor

bcmills commented Jun 12, 2018

byte array contains bytes from zlib deflate function output

Can you give a command line or program that produces that output?

I get a different output using zlib-flate on my machine:

~$ echo -n 'Hello Hello Hello Hello Hello Hello!' | zlib-flate -compress | hexdump -v -e '/1 "%1u, "'
120, 156, 243, 72, 205, 201, 201, 87, 240, 192, 71, 42, 2, 0, 232, 207, 12, 122,

However, even with that input I can reproduce the “corrupt input before offset” error:
https://play.golang.org/p/ohVxnLuOact

@bcmills
Copy link
Contributor

bcmills commented Jun 12, 2018

CC @dsnet @mdempsky for compress/flate.

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 12, 2018
@bcmills bcmills added this to the Go1.11 milestone Jun 12, 2018
@bcmills
Copy link
Contributor

bcmills commented Jun 12, 2018

I can confirm that zlib-flate -uncompress interprets your original encoding as the expected string too:

~$ echo -ne 'x\xda\xf3H\xcd\xc9\xc9W\xf0\xc0G*2' | zlib-flate -uncompress
Hello Hello Hello Hello Hello Hello!

@dsnet
Copy link
Member

dsnet commented Jun 12, 2018

This is not a DEFLATE stream, but rather is a ZLIB stream. Changing the decompressor from DEFLATE to ZLIB decompresses the data: https://play.golang.org/p/AJLlyc8H_jr

Both DELFATE and ZLIB lack magic numbers, so it is impossible to distinguish between the two with perfect accurancy. However, the data stream here starts with 0x78, which is a very common for a ZLIB stream as it indicates that the DEFLATE was used under the hood with a window size of 32KiB.

@dsnet dsnet closed this as completed Jun 12, 2018
@golang golang locked and limited conversation to collaborators Jun 12, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.11
Development

No branches or pull requests
4 participants
@zeslava @bcmills @dsnet @gopherbot