Comment 1:

The allocation size (size of the underlying array) of a slice of 240000 strings is
5.7MB. Not having these megabytes already written in the binary (to be directly mapped)
means generating them at runtime and slowing down program startup.

Comment 2:

you have 235886 strings, total string size (not including overhead) is 2257223.
Then the bundledDictionary slice has 235886 entry, each entry contains a
StringHeader, which is 16-byte on amd64, so you will need to add at least
235886 * 16 bytes of overhead (which is 3774176 bytes).
so actually you bundledDictionary will use at least 7918487 bytes of space,
far larger than the string data themselves.
I suggest you concatenate the strings to make a bigger one, and split at
runtime (or save the index of each string; or even better, use trie to store
the strings, so that it doesn't need any initialization at runtime)
PS: the reason why Go binary uses ~4MB more is that every string is actually
stored as a separate StringHeader in the Go string table, so each strings adds
further 16 bytes of overhead (3774176 bytes). Taking that into account,
now the binary uses 11692663 bytes to store all the strings and the slice,
and this number explains 11MB increase in binary size.
I don't think we can do anything here. Don't use too many strings.

Status changed to WorkingAsIntended.

Comment 3:

I don't think we need to store string constants as StringHeaders. The overhead should
happen only once.

Comment 4:

ok.
That means the linker must keep track of which stringheader is referenced,
seems doable but it will need big changes to go.string.* section.
but perhaps we should first solve the 1.5GB RSS problem of 6g first.
(also cmd/6g will generate a 33040637 bytes rw.6!)
also, it turns out our go.string.* has fairly big space inefficiency problem.
each Go string is stored like this in the binary in the go.string.* section:
  pointer *byte // points to the payload, always this address + sizeof(uintptr)*2, redundant
  len uintptr // not redundant, but the compiler could almost always inline this number.
  payload byte[len+1] // always has 0-termination. why??
  padding byte[padlen] // pad to next uintptr-aligned for next StringHeader.
which means that each (8*n+k) byte string will take (8-k) bytes more.
(e.g. a 8 byte string will take 8 extra bytes of padding. Yes. you read this correctly.)
The problem seems to be just don't store StringHeader and always either
construct in runtime or (in this case) store in the temporary statictmp_0020.
which means we can concatenate all strings in the go.string.* section,
and this also enables substring reuse optimization (e.g. if there are both "a" and "ab"
strings in a Go program, we only store "ab" in the string section)
Too big a change for the current linker, let's consider this when cmd/link takes over.

Labels changed: added release-go1.4, repo-main.

Status changed to Accepted.

Comment 5:

Josh (copied) was looking into this recently.

Comment 6:

I came to the same conclusion as minux, that is a significant change and should wait for
cmd/link. There are definitely non-trivial binary size savings to be had here for all
programs; even hello world contains lots of small strings.

Comment 7:

I don't know if this also belong here:
https://github.com/fiam/gounidecode
It have big map in https://github.com/fiam/gounidecode/blob/master/unidecode/table.go:
`var transliterations = map[rune]string`
Over 750KB input file make output binary bigger by almost 5 MB.
Fell free to remove this comment.
Best regards,
Dobrosław Żybort

Comment 8:

Labels changed: added release-go1.5, removed release-go1.4.

CL https://golang.org/cl/11698 mentions this issue.