cmd/compile: rc1.14rc1 build internal/ssa out of memory #37122
Comments
cagedmantis
added
the
NeedsInvestigation
cagedmantis
changed the title
What does "plenty" mean here? I don't know how quickly top refreshes, but building package ssa eats ram (and potentially dies) very quickly. I suspect this is probably a duplicate of #27739. I have a series of CLs ready for 1.15 that will shrink package ssa by about 25%. |
|
The machine has 12G RAM. Top showed memory usage varying between 250MB and
500MB most of the time, once getting up to about 1GB, but never anywhere
close to running out of physical memory. I didn't do any ulimit or other
special action, so could be there was some artificial software limit
exceeded.
…On Fri, Feb 7, 2020 at 12:30 PM Josh Bleecher Snyder < ***@***.***> wrote:
There was actually plenty of available RAM at the time, according to top.
What does "plenty" mean here? I don't know how quickly top refreshes, but
building package ssa eats ram (and potentially dies) very quickly.
I suspect this is probably a duplicate of #27739
<#27739>. I have a series of CLs ready
for 1.15 that will shrink package ssa by about 25%.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#37122?email_source=notifications&email_token=ACADPOUMWWDAUO4Q46AAAILRBXAFDA5CNFSM4KRT4L52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELEP2QI#issuecomment-583597377>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACADPOWHJ7XWMVXDU3JSO5DRBXAFDANCNFSM4KRT4L5Q>
.
|
|
Yeah, 12gb really ought to be enough. :) Does it reproduce consistently? cc @mknyszek @aclements just in case |
|
Yes, reproduces 100%. Happy to re-run with any additional debugging you can teach me. In any event, if a normal Go install requires tons of memory, it is probably worth measuring this at release candidate build time and adding a comment to the install doc, or even better finding a way to determine at runtime that the process is likely to fail for a known reason. |
|
Also, just reproduced it on another OpenBSD machine running stable+syspatch, so this is probably not some recent OpenBSD issue. I'm happy to give Austin a login if constructing such a machine would be burdensome to y'all. I know I'm running on a niche OS and aspire to the skill to give you a better bug report. |
|
To rule out some things, let's try disabling parallel and concurrent compilation:
If that still fails, let's see how much RAM the shell thinks it is using:
|
|
OpenBSD by default has a very low RLIMIT_DATA. What's the output of Even if RLIMIT_DATA is raised, OpenBSD limits anonymous mappings to a fairly small factor larger than physical memory (even if they're untouched), and Go 1.14 does create some larger sparse mappings than Go 1.13 did. For this limit, the relevant top column to watch would be VSZ, I believe. |
Sorry, |
|
"ulimit -a" on this 16GB physical NUQkit reports stack 8MB, lockedmem 5GB, mem 16GB. GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash Maybe I'm blind, but |
|
Sounds like Austin probably has a better bead on what’s going on here. I’m curious though whether my planned changes will help sufficiently. To test,
and then try again. |
|
Hurray and thank you! With your change, builds and tests successfully. |
|
That’s good to hear. That won’t land until the 1.15 release, though. |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env)?See attached file.
What did you do?
What did you expect to see?
compile and test results as usual
What did you see instead?
There was actually plenty of available RAM at the time, according to top.
go1.14rc1.txt
The text was updated successfully, but these errors were encountered: