-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/compile: init of large maps of pointers allocates too much memory #51543
Comments
So both 1.16.x and 1.18 - master affected by this "bug"? |
@mengzhuo it seems your quick test compiles the code generator instead of the generated code. And yes, 1.18 also has the issue. |
CC @randall77 |
@heschi @dr2chase |
I think I have a fix, waiting to see if it breaks tests. Might speed up compilation a bit, in general. |
In the case of this example code, 1.15 simply didn't do the inlines. |
The cause is I think an inefficient set representation. The output is O(N) but I am pretty sure we construct O(N) information for O(N) blocks along the way. My go-to hammer for this problem would be an applicative balanced tree set implementation, to see if that knocked the complexity down. |
Change https://go.dev/cl/397318 mentions this issue: |
What version of Go are you using (
go version
)?1.17, the bug appeared in 1.16.0.
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
In one of our systems we had a large generated map containing some structs with pointers to booleans. Since it is not possible to express a pointer to a boolean literal, we had to use a simple function wrapper like this:
It worked just fine, until we updated to a more recent version of Go.
Starting with Go 1.16.0 this thing causes to the compiler to be OOM-killed, I believe because of some changes in the inlining logic introduced in that version. If you mark the
newBool
function with//go:noinline
at the top, the compiler doesn't run out of memory.Here is a little generator that creates two programs
good.go
andbad.go
, the former with the//go:noinline
directive and the latter without.The resulting programs look like this:
I run the compiler like this to quickly try different Go versions:
What did you expect to see?
Expected it to compile quickly and successfully.
What did you see instead?
Go compiler versions 1.16.0+ spends tens of seconds and gigabytes of memory, eventually getting OOM-killed.
The text was updated successfully, but these errors were encountered: