cmd/compile: objects emit generic instantiations for imported packages #56718
Labels
compiler/runtime
Issues related to the Go compiler and/or runtime.
NeedsFix
The path to resolution is known, but the work has not been done.
Milestone
With the use of atomic.Pointer in package sync, this program is now generating the full compiled code for atomic.Pointer into its object file even though it's not being used at all:
The part I didn't omit with ... is all that should be here, plus maybe a package init. The omitted part appears to be the entire compilation of four different instantiations of sync/atomic.Pointer, plus all their supporting type declarations:
Given that there is no code being emitted for this package at all, there's no need to emit all of this. (It's possible they are referenced from the definition of
go:info.sync.Map
, but that is not being emitted here either. The generated code correctly assumes that will be provided by package sync's object file.)Overall the size increase is 33X since Go 1.19:
Obviously there is a small denominator in that 33X, but there is also very little use of generics yet.
There is a deeper problem than dead code here too.
When we were talking about instantiation in the early days of generics, one of the ways we discussed to avoid code size issues would be for packages to know which instantiations they use are already present in packages they import, so that definitions would not need to be generated repeatedly in multiple packages. For example, since package sync generated the definition for atomic.Pointer[any] any program directly or indirectly importing package sync can use atomic.Pointer[any] “for free” without itself regenerating that code. Of course, there's a search problem for answering the question of whether any package in the transitive import closure has already generated the code. But at the least, if we see a type in the ordinary course of reading export data for imported packages, we should mark it as not needed to be regenerated: anything in export data is handled by the package that wrote the export data. Applying that rule has no search problem.
In the standard library we are barely starting to use generics, yet we have 15 packages whose objects repeat the atomic.Pointer code from sync, 13 that repeat atomic.Pointer code from go/token, 9 that repeat an atomic.Pointer[string] from internal/godebug (in my working tree), and three that repeat an atomic.Pointer[response] from net/http:
For what it's worth, the objects we shipped in Go 1.19 appear to have the same duplication of atomic.Pointer[go/token.File], so the bug is not new.
15 doesn't look so bad, but when compiling Kubernetes we end up with 122 copies each of four different atomic.Pointer instantiations from package sync:
If this is easy to fix it would be good to do it for Go 1.20. If not, then since Go 1.19 appears to behave the same way, it's probably not strictly a release blocker.
The text was updated successfully, but these errors were encountered: