Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/image/bmp: out of memory #10399

Closed
dvyukov opened this issue Apr 9, 2015 · 6 comments
Closed

x/image/bmp: out of memory #10399

dvyukov opened this issue Apr 9, 2015 · 6 comments

Comments

@dvyukov
Copy link
Member

dvyukov commented Apr 9, 2015

Run the following program on the following input:

package main

import (
    "bytes"
    "io/ioutil"
    "os"
    "golang.org/x/image/bmp"
)

func main() {
    data, _ := ioutil.ReadFile(os.Args[1])
    img, err := bmp.Decode(bytes.NewReader(data))
    if err != nil {
        return
    }
    var w bytes.Buffer
    err = bmp.Encode(&w, img)
    if err != nil {
        panic(err)
    }
}

https://drive.google.com/file/d/0B20Uwp8Hs1oCTzFrMndWMjNtTHM/view?usp=sharing

It crashes as:

fatal error: runtime: out of memory

goroutine 1 [running]:
runtime.systemstack_switch()
    src/runtime/asm_amd64.s:216 fp=0xc208041b58 sp=0xc208041b50
runtime.mallocgc(0x1000000300, 0x4b27c0, 0x1, 0x0)
    src/runtime/malloc.go:625 +0x8f1 fp=0xc208041c28 sp=0xc208041b58
runtime.newarray(0x4b27c0, 0x1000000300, 0x1)
    src/runtime/malloc.go:736 +0xce fp=0xc208041c68 sp=0xc208041c28
runtime.makeslice(0x4ac5a0, 0x1000000300, 0x1000000300, 0x0, 0x0, 0x0)
    src/runtime/slice.go:32 +0x178 fp=0xc208041cb8 sp=0xc208041c68
golang.org/x/image/bmp.decodeNRGBA(0x7f87732092d8, 0xc208014420, 0x7f87732090d0, 0xc20802e020, 0x10, 0x4000000c, 0x1, 0x0, 0x0, 0x0, ...)
    src/golang.org/x/image/bmp/reader.go:83 +0x185 fp=0xc208041e30 sp=0xc208041cb8
golang.org/x/image/bmp.Decode(0x7f87732092d8, 0xc208014420, 0x0, 0x0, 0x0, 0x0)
    src/golang.org/x/image/bmp/reader.go:114 +0x1be fp=0xc208041ed0 sp=0xc208041e30
main.main()
    /tmp/bmp.go:12 +0x132 fp=0xc208041f90 sp=0xc208041ed0

I am on commit 65a798f031fd31a65574938bed2ec44c2bcba496

@dvyukov dvyukov added this to the Go1.5 milestone Apr 9, 2015
@chai2010
Copy link
Contributor

The test bmp size is 16x1073741836, bpp is 32.
This cause the image.NewNRGBA(image.Rect(0, 0, c.Width, c.Height)) panic.
I think this is not a bug.

@dvyukov
Copy link
Member Author

dvyukov commented Apr 23, 2015

My understanding is that BMP is not compressed format. So decoding must try to allocate 16x1073741836x4 only if input size is larger than that. In this case input is few bytes, so an attempt to allocate 64GB of memory does look like a bug.
FWIW, encoding/gob has such logic that detects when declared data size is larger than input.
Ideally we have such logic for compressed image formats as well. But it is not directly possible. Probably we can introduce MaxMem option for decoding of compressed formats, i.e. "decode this blob using no more than 100MB of memory".

@minux
Copy link
Member

minux commented Apr 23, 2015 via email

@chai2010
Copy link
Contributor

@minux x/image/bmp does not supports any compression.

@rsc
Copy link
Contributor

rsc commented Apr 26, 2015

@dvyukov, use Go 1.5 milestone for bugs in released code. Subrepos are not released. The code in x/image/bmp and Go 1.5 are completely unrelated. The milestone for subrepos is Unreleased.

@rsc rsc modified the milestones: Unreleased, Go1.5 Apr 26, 2015
@nigeltao
Copy link
Contributor

BMP is not a compressed format, but the structure of the file format is first header, then payload. The image dimensions are in the header, and the decoder allocates a pixel buffer after the header is processed but before the payload is processed, because processing the payload involves writing the pixels to somewhere: the pixel buffer.

Decode takes an io.Reader, not an io.ReadSeeker, so at the time it allocates the pixel buffer, it cannot know whether or not there's 'enough input' remaining. Conversely, we may be decoding an image downloaded from a socket, where we cannot know how much input is remaining. So I think this is all WAI.

For decoding images in general, with compression, knowing the total file size doesn't necessarily mean that you know whether or not there's enough pixel data. LZW-style length-difference or RLE encoding means that very large uncompressed data can compress very, very well.

That the various image libraries should reject an 16x1073741836 early, checking some sort of MaxMem option before allocating a buffer, is issue #5050.

@golang golang locked and limited conversation to collaborators Jun 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants