runtime: potential memory leak in Go1.11.2 #29185

andig · 2018-12-12T08:21:07Z

What version of Go are you using (`go version`)?

$ go version
go version go1.11.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/andig/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/andig/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.2/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/andig/htdocs/ingress/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/73/89ycv7qn51j4kbm04jsz9b840000gn/T/go-build037396939=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I realize this is very unlikely but I wouldn't expect this simple loop to show increasing memory usage:

for {
	time.Sleep(time.Second)
	runtime.GC() // this is optional
	var memstats runtime.MemStats
	runtime.ReadMemStats(&memstats)
	fmt.Printf("%db\n", memstats.Alloc)
}

What did you expect to see?

Consant memory usage

What did you see instead?

Rising memory:

❯ go run cmd/test/main.go
68360b
...
68376b
...
68832b
...
69288b
...
69744b
...
71536b
...
73328b

Omitting the GC() call makes the memory rise quicker in smaller steps.

The text was updated successfully, but these errors were encountered:

networkimprov · 2018-12-13T00:26:25Z

How many iterations have you tried?

mark-rushakoff · 2018-12-13T00:57:50Z

With this repro on go version devel +9a11b73635 Wed Dec 12 15:00:01 2018 +0000 darwin/amd64:

package main

import (
	"fmt"
	"runtime"
	"time"
)

func main() {
	for {
		time.Sleep(time.Second)
		// runtime.GC() // this is optional
		var memstats runtime.MemStats
		runtime.ReadMemStats(&memstats)
		// println(memstats.Alloc)
		fmt.Printf("%db\n", memstats.Alloc)
	}
}

And running go build then running the binary with GODEBUG=allocfreetrace=1, I see repeated output indicating that the allocations are happening in fmt.Printf (converting memstats.Alloc to an interface, and a defer while writing to stdout):

73688b
tracealloc(0xc00007e2d0, 0x10, runtime.uint64InterfacePtr)
goroutine 1 [running]:
runtime.mallocgc(0x10, 0x10a46a0, 0x104d400, 0xc00008e8e8)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/malloc.go:1011 +0x4ae fp=0xc00008e868 sp=0xc00008e7c8 pc=0x100a6de
runtime.convT64(0x11fe8, 0xc000086000)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/iface.go:345 +0x5b fp=0xc00008e898 sp=0xc00008e868 pc=0x10085bb
main.main()
	/tmp/m.go:16 +0x80 fp=0xc00008ff98 sp=0xc00008e898 pc=0x1093870
runtime.main()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/proc.go:200 +0x204 fp=0xc00008ffe0 sp=0xc00008ff98 pc=0x1029414
runtime.goexit()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x1050b91

tracealloc(0xc0000805c0, 0x40, runtime._defer)
goroutine 1 [running]:
runtime.systemstack_switch()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/asm_amd64.s:311 fp=0xc00008e6a8 sp=0xc00008e6a0 pc=0x104ec60
runtime.newdefer(0x80000000008, 0x7ffff80000000000)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/panic.go:240 +0x17a fp=0xc00008e718 sp=0xc00008e6a8 pc=0x1026d4a
runtime.deferproc(0xc000000008, 0x10cda70)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/panic.go:107 +0x3e fp=0xc00008e748 sp=0xc00008e718 pc=0x102681e
internal/poll.(*FD).Write(0xc000088000, 0xc00007e018, 0x7, 0x8, 0x0, 0x0, 0x0)
	/Users/mr/gotip/src/github.com/golang/go/src/internal/poll/fd_unix.go:258 +0x87 fp=0xc00008e7b0 sp=0xc00008e748 pc=0x1084c87
os.(*File).write(...)
	/Users/mr/gotip/src/github.com/golang/go/src/os/file_unix.go:266
os.(*File).Write(0xc000086000, 0xc00007e018, 0x7, 0x8, 0x1, 0x1, 0x10085bb)
	/Users/mr/gotip/src/github.com/golang/go/src/os/file.go:145 +0x74 fp=0xc00008e830 sp=0xc00008e7b0 pc=0x1085f24
fmt.Fprintf(0x10ddf20, 0xc000086000, 0x10c6b4c, 0x4, 0xc00008ff78, 0x1, 0x1, 0x7, 0x0, 0x0)
	/Users/mr/gotip/src/github.com/golang/go/src/fmt/print.go:200 +0xa5 fp=0xc00008e898 sp=0xc00008e830 pc=0x108bcb5
fmt.Printf(...)
	/Users/mr/gotip/src/github.com/golang/go/src/fmt/print.go:208
main.main()
	/tmp/m.go:16 +0xf7 fp=0xc00008ff98 sp=0xc00008e898 pc=0x10938e7
runtime.main()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/proc.go:200 +0x204 fp=0xc00008ffe0 sp=0xc00008ff98 pc=0x1029414
runtime.goexit()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x1050b91

But if I comment out the fmt.Printf and uncomment the println, the allocations immediately steady out and the same value of memstats.Alloc are printed, at least for the first minute or two.

I really doubt fmt.Printf is actually leaking memory. Is it just the runtime allowing space for defer and interface conversion to grow to a certain size before garbage collecting that area?

kaypour · 2018-12-13T16:42:34Z

Is it because var memstats runtime.MemStats is heap allocated when you use fmt.Printf?

Using fmt.Printf

$ go build -gcflags '-m' main.go
# command-line-arguments
./main.go:16:31: memstats.Alloc escapes to heap
./main.go:14:24: main &memstats does not escape
./main.go:16:13: main ... argument does not escape

Using println

$ go build -gcflags '-m' main.go
# command-line-arguments
./main.go:13:24: main &memstats does not escape

I think it is common for the escape analysis to allocate on the heap, if the function takes in an interface type.

andig · 2018-12-14T13:20:55Z

Thank you all for the detailed feedback! From what I've understood this is the expected behaviour (memory increase until GC kicks in). Closing for now, please let me know if I did misunderstand the feedback.

ALTree changed the title ~~1.11.2 Potential memory leak~~ runtime: potential memory leak in Go1.11.2 Dec 12, 2018

ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 12, 2018

ALTree added this to the Go1.12 milestone Dec 12, 2018

andig closed this as completed Dec 14, 2018

golang locked and limited conversation to collaborators Dec 14, 2019

gopherbot added the FrozenDueToAge label Dec 14, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime: potential memory leak in Go1.11.2 #29185

runtime: potential memory leak in Go1.11.2 #29185

andig commented Dec 12, 2018

networkimprov commented Dec 13, 2018 •

edited

mark-rushakoff commented Dec 13, 2018

kaypour commented Dec 13, 2018 •

edited

andig commented Dec 14, 2018

runtime: potential memory leak in Go1.11.2 #29185

runtime: potential memory leak in Go1.11.2 #29185

Comments

andig commented Dec 12, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

networkimprov commented Dec 13, 2018 • edited

mark-rushakoff commented Dec 13, 2018

kaypour commented Dec 13, 2018 • edited

andig commented Dec 14, 2018

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

networkimprov commented Dec 13, 2018 •

edited

kaypour commented Dec 13, 2018 •

edited