Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image/png: limit memory Decode can use while decoding #12512

Closed
ghost opened this issue Sep 5, 2015 · 6 comments
Closed

image/png: limit memory Decode can use while decoding #12512

ghost opened this issue Sep 5, 2015 · 6 comments

Comments

@ghost
Copy link

ghost commented Sep 5, 2015

With the DEFLATE compression used by the PNG image format one can compress a 50 gigapixel image to a 6 Mb PNG file. When decoding such a file, png.Decode throws a fatal error: runtime: out of memory.

image/png should provide a safety switch that allows the user to limit the amount of memory that png.Decode will use.

Test case: https://github.com/opennota/spark

Reference: https://www.bamsoftware.com/hacks/deflate.html

@bradfitz
Copy link
Contributor

bradfitz commented Sep 5, 2015

It does have such a safety switch already: http://golang.org/pkg/image/png/#DecodeConfig

I'm not sure there's anything to be done here that wouldn't be redundant with that. You could imagine a new Decoder type with some optional limit options, and have Decode return an error if limits are violated, I guess. That might be a little more friendly.

/cc @nigeltao @mpl

@bradfitz bradfitz changed the title Limit memory png.Decode can use while decoding image/png: limit memory Decode can use while decoding Sep 5, 2015
@bradfitz
Copy link
Contributor

bradfitz commented Sep 5, 2015

(this applies to all image decoders, really... not just png)

@ghost
Copy link
Author

ghost commented Sep 5, 2015

It does have such a safety switch already: http://golang.org/pkg/image/png/#DecodeConfig

Ah, yes. It seems there's no way to continue reading the entire image from the same reader without seeking to the start of the stream, though.

@bradfitz
Copy link
Contributor

bradfitz commented Sep 5, 2015

There is a way, once you learn the rest of the standard library. :-)

Imagine you have a streamReader which is not seekable. (e.g. it's coming from the network, via an HTTP request or response or whatever). You can read the header, verify its properties, and then decode the rest of the image with the already-read header re-stitched on to the front like this:

var header bytes.Buffer
conf, err := png.DecodeConfig(io.TeeReader(streamReader, &header))
if err != nil || !confValid(conf) { ... }
im, err := png.Decode(io.MultiReader(&header, streamReader)

See the docs for io.TeeReader and io.MultiReader to see how that works.

@ghost
Copy link
Author

ghost commented Sep 6, 2015

@bradfitz
Wow, cool. I think the issue can be closed, then.

@minux minux closed this as completed Sep 6, 2015
@nigeltao
Copy link
Contributor

nigeltao commented Sep 7, 2015

Yeah, what @bradfitz said.

As for a nicer API, this is issue #5050.

@golang golang locked and limited conversation to collaborators Sep 6, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants