You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was looking recently into compiler allocations, and because I wanted to know the breakdown by type, I hacked up tracealloc and friends in mprof.go to print exactly one line per allocation and nothing else (including suppressing output in tracefree and tracegc). Surprisingly, this output differs significantly from that reported by the memory profiler, even at runtime.MemProfileRate=1.
For example, running go tool 6g -memprofile=mem.prof html/template/*.go and then running go tool pprof -alloc_objects yields a count of 458,794 allocations, almost none of which are from cmd/internal/gc.walkexpr.
Setting runtime.MemProfileRate=1 and repeating the experiment turns up 607,429 allocations, 5.87% of which are from cmd/internal/gc.walkexpr. (The np **Node parameter escapes, causing an allocation of a *Node, which is only 8 bytes, thus the previous significant undercounting.)
Running with GODEBUG=allocfreetrace=1 go tool 6g html/template/* 2>&1 | grep "tracealloc" | wc -l yields 1,032,392 allocations, which is 70% more than reported by the memory profiler! This number is very stable across repeated runs, with variation less than 0.01%. Only logging allocs with size > 16 in tracealloc yields a count of 740,816 allocations, so this is probably not a tiny alloc issue.
There are probably multiple things to do here:
Understand the mismatch and fix the memory profiler.
Add a memprofilerate flag to the compiler.
Warn in pprof when -alloc_objects is used with memprofilerate != 1, because allocation counts can be dramatically skewed by the size of the allocation.
Also call runtime.GC before exit to ensure
that the profiler picks up all allocations.
Fixesgolang#10537.
Change-Id: Ibfbfc88652ac0ce30a6d1ae392f919df6c1e8126
Also call runtime.GC before exit to ensure
that the profiler picks up all allocations.
Fixesgolang#10537.
Change-Id: Ibfbfc88652ac0ce30a6d1ae392f919df6c1e8126
Apologies in advance that this is vague.
I was looking recently into compiler allocations, and because I wanted to know the breakdown by type, I hacked up
tracealloc
and friends inmprof.go
to print exactly one line per allocation and nothing else (including suppressing output intracefree
andtracegc
). Surprisingly, this output differs significantly from that reported by the memory profiler, even atruntime.MemProfileRate=1
.For example, running
go tool 6g -memprofile=mem.prof html/template/*.go
and then runninggo tool pprof -alloc_objects
yields a count of 458,794 allocations, almost none of which are fromcmd/internal/gc.walkexpr
.Setting
runtime.MemProfileRate=1
and repeating the experiment turns up 607,429 allocations, 5.87% of which are fromcmd/internal/gc.walkexpr
. (Thenp **Node
parameter escapes, causing an allocation of a*Node
, which is only 8 bytes, thus the previous significant undercounting.)Running with
GODEBUG=allocfreetrace=1 go tool 6g html/template/* 2>&1 | grep "tracealloc" | wc -l
yields 1,032,392 allocations, which is 70% more than reported by the memory profiler! This number is very stable across repeated runs, with variation less than 0.01%. Only logging allocs with size > 16 intracealloc
yields a count of 740,816 allocations, so this is probably not a tiny alloc issue.There are probably multiple things to do here:
-alloc_objects
is used with memprofilerate != 1, because allocation counts can be dramatically skewed by the size of the allocation./cc @rsc @dr2chase @dvyukov
The text was updated successfully, but these errors were encountered: