Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

image/gif: decoder calls defer in a loop for every frame affecting memory usage #22237

Closed
artyom opened this issue Oct 12, 2017 · 1 comment
Closed

image/gif: decoder calls defer in a loop for every frame affecting memory usage #22237

artyom opened this issue Oct 12, 2017 · 1 comment
Labels
FrozenDueToAge

Comments

@artyom
Copy link
Member

artyom commented Oct 12, 2017

Please answer these questions before submitting your issue. Thanks!

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Save https://play.golang.org/p/l7FgKFL6l0 to the gif_test.go file along with this attachment (save it under 1x1-40k.gif name) which is a 1x1 px 40k frames GIF.

Run test:

go test

What did you expect to see?

Test passes.

What did you see instead?

--- FAIL: TestDecodMemoryConsumption (1.35s)
	gif_test.go:24: Decode increased heap by 1014MB
FAIL

This happens because gif.decoder.decode() calls defers in a loop, causing multiple *lzw.decoders to stay referenced until decode() returns, thus increasing heap size proportionally to the number of frames:

go/src/image/gif/reader.go

Lines 226 to 227 in 764a6ac

lzwr := lzw.NewReader(br, lzw.LSB, int(litWidth))
defer lzwr.Close()

I plan to submit a CL.

System details

go version go1.9.1 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/tmp/go:/Users/artyom/go"
GORACE=""
GOROOT="/Users/artyom/Library/go"
GOTOOLDIR="/Users/artyom/Library/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lb/3rk8rqs53czgb4v35w_342xc0000gn/T/go-build010192856=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version go1.9.1 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.9.1
uname -v: Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13
BuildVersion:	17A405
lldb --version: lldb-900.0.45
  Swift-4.0
@gopherbot
Copy link

Change https://golang.org/cl/70370 mentions this issue: image/gif: avoid setting defers in the decode loop

@golang golang locked and limited conversation to collaborators Oct 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@artyom @gopherbot