cmd/compile: significant binary size increase in go1.11 #27266
Comments
|
And go1.11 consumes more resources, there are some issue (#26186 & #26132 ) tracker this. Use go1.10.3 to compile normally, but switching to go1.11 will give an error. |
josharian
changed the title
josharian
added
the
NeedsInvestigation
|
Hi @josharian the sizecmp dependon otool, which is the tools for OSX, and can't run on my linux env: Use bloaty to analyse the binaries: |
|
The size increase is not just DWARF information, I rebuild the file with flag GOLDFLAGS="-s -w" (trip DWARF, debuginfo, symbol). As shown in the following data, the size increased about 16%. |
|
Our internal project also compiled with -s -w: (The debug binaries are a tad bit smaller in 1.11 tan those from 1.10, but that's due to DWARF compression) 1.11 also ~14% larger images
|
|
From #27784 (comment): "bloaty suggests the extra size is coming from __gopclntab" |
|
The bloaty analyse result by "make all WHAT=cmd/hyperkube": |
|
The bloaty analyse result by 'make all WHAT=cmd/hyperkube GOLDFLAGS="-s -w"' (trip DWARF, debuginfo, symbol): |
|
I'll also kindly rope in some more folks who might be interested in this bug @randall77 @ianlancetaylor @bradfitz @cherrymui @heschik |
|
/cc @dr2chase @aclements |
|
I haven't looked into this deeply at all, but growth in .gopclntab would seem to implicate https://golang.org/issue/24543 (safepoints everywhere). The design doc there says to expect growth of ~10% unless optimizations are applied. |
|
The growth in .gopclntab is almost certainly mostly from safepoints everywhere, but that's only 13.5MB of growth, or 6.25% of overall growth in the unstripped binary and 10% growth in the stripped binary. Here's a section-by-section comparison from benchstat of the bloaty output: For unstripped binaries: For .gopclntab, it would be reasonable to strip the safepoints everywhere data in |
|
1.12 once again increases binary size on the 1.11. 1.11
1.12 beta1
~600k increase on stripped binary. :/ and ~1,2M on nonstripped one |
This comment has been minimized.
This comment has been minimized.
|
I compiled a hello world go app the other day and noticed the binary had functions for cuneiform. While five thousand years of backwards compatibility is commendable, I'd expected that its linker be capable of exercising greater parsimony. |
|
@jart, the compiler & linker do a lot in that area, but it often stops being great once it sees an interface value. It does little to no proving to itself what concrete types of values can go in interfaces. There are several open bugs in that area, but they're all tricky. If you want to file new bugs with concrete examples (after searching for existing dups), that'd be helpful. This bug is more of a meta bug. |
|
Found this with a binary of mine that includes a more complicated set of parsers: $ bloaty polyglot-go1.11.5
VM SIZE FILE SIZE
-------------- --------------
61.5% 26.7Mi __TEXT,__text 26.7Mi 48.5%
0.0% 0 [Unmapped] 11.9Mi 21.5%
26.4% 11.4Mi __TEXT,__gopclntab 11.4Mi 20.8%
8.6% 3.72Mi __TEXT,__rodata 3.72Mi 6.7%
1.9% 823Ki String Table 823Ki 1.5%
0.8% 338Ki Symbol Table 338Ki 0.6%
0.4% 179Ki __DATA,__noptrdata 179Ki 0.3%
0.3% 126Ki __DATA,__bss 0 0.0%
0.1% 58.5Ki __DATA,__data 58.5Ki 0.1%
0.0% 13.7Ki __TEXT,__typelink 13.7Ki 0.0%
0.0% 12.5Ki __DATA,__noptrbss 0 0.0%
0.0% 3.24Ki __TEXT,__itablink 3.24Ki 0.0%
0.0% 2.50Ki [Mach-O Headers] 2.60Ki 0.0%
0.0% 2.48Ki [__TEXT] 2.38Ki 0.0%
0.0% 688 __DATA,__nl_symbol_ptr 688 0.0%
0.0% 676 Indirect Symbol Table 676 0.0%
0.0% 498 __TEXT,__symbol_stub1 498 0.0%
0.0% 79 [__DATA] 63 0.0%
100.0% 43.4Mi TOTAL 55.1Mi 100.0%
$ bloaty polyglot-go1.12beta2
VM SIZE FILE SIZE
-------------- --------------
59.3% 26.7Mi __TEXT,__text 26.7Mi 45.8%
0.0% 0 [Unmapped] 13.5Mi 23.1%
26.7% 12.0Mi __TEXT,__gopclntab 12.0Mi 20.6%
9.5% 4.28Mi __TEXT,__rodata 4.28Mi 7.3%
2.6% 1.19Mi String Table 1.19Mi 2.0%
1.0% 455Ki Symbol Table 455Ki 0.8%
0.4% 179Ki __DATA,__noptrdata 179Ki 0.3%
0.3% 122Ki __DATA,__bss 0 0.0%
0.1% 58.8Ki __DATA,__data 58.8Ki 0.1%
0.0% 14.0Ki __TEXT,__typelink 14.0Ki 0.0%
0.0% 12.6Ki __DATA,__noptrbss 0 0.0%
0.0% 5.06Ki [__TEXT] 4.96Ki 0.0%
0.0% 3.37Ki __TEXT,__itablink 3.37Ki 0.0%
0.0% 2.50Ki [Mach-O Headers] 2.60Ki 0.0%
0.0% 1.02Ki __DATA,__nl_symbol_ptr 1.02Ki 0.0%
0.0% 1.00Ki Indirect Symbol Table 1.00Ki 0.0%
0.0% 756 __TEXT,__symbol_stub1 756 0.0%
0.0% 71 [__DATA] 55 0.0%
100.0% 45.1Mi TOTAL 58.4Mi 100.0%A 3 MB increase. |
|
I see that |
|
FYI, there is a new discussion of gopclntab in #36313 |
|
/label binary-size |
|
This bug seems pretty non-actionable (we already have a meta bug, linked above, and a bunch of specific bugs). It doesn't help to have another meta bug open. I suspected the Go 1.15 improvements (stack maps, register maps, linker, etc) fixed this anyway, so I went to measure on a big program. Kubernetes is too ugly to build (I wanted something that just used Go modules & worked back to Go 1.11) so took @Xe's program above (#27266 (comment)) and removed a couple lines from it to make it build. I get: I didn't measure Go 1.10 because it doesn't support Go modules and I couldn't be bothered to fight a GOPATH. But let's assume Go 1.10 was a bit smaller than Go 1.11. Go 1.12 was certainly worse than Go 1.11, but Go 1.14 got better than Go 1.11, and then Go 1.15(-dev) is the best yet. Maybe it's even down to Go 1.10 size. If somebody has a self-contained large program that builds in all versions, let us know. In any case, I'm going to close this bug as fixed, at least enough. The plenty of others bugs labeled binary-size track more specific improvements we could make. |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yep, upgrade the golang to version 1.11 and build the software.
What operating system and processor architecture are you using (
go env)?What did you do?
What did you expect to see?
The compiler version upgrade has little effect on the size of the generated binary.
What did you see instead?
The size increased 26.7% compared to 1.10.x versions.
The text was updated successfully, but these errors were encountered: