-
Notifications
You must be signed in to change notification settings - Fork 18k
runtime: free GC workbuf memory #19325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
CL https://golang.org/cl/38581 mentions this issue. |
CL https://golang.org/cl/38577 mentions this issue. |
CL https://golang.org/cl/38580 mentions this issue. |
CL https://golang.org/cl/38578 mentions this issue. |
CL https://golang.org/cl/38582 mentions this issue. |
CL https://golang.org/cl/38576 mentions this issue. |
CL https://golang.org/cl/38574 mentions this issue. |
CL https://golang.org/cl/38579 mentions this issue. |
CL https://golang.org/cl/38575 mentions this issue. |
We're about to generalize _MSpanStack to be used for other forms of in-heap manual memory management in the runtime. This is an automated rename of _MSpanStack to _MSpanManual plus some comment fix-ups. For #19325. Change-Id: I1e20a57bb3b87a0d324382f92a3e294ffc767395 Reviewed-on: https://go-review.googlesource.com/38574 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
We're going to use this free list for other types of manually-managed memory in the heap. For #19325. Change-Id: Ib7e682295133eabfddf3a84f44db43d937bfdd9c Reviewed-on: https://go-review.googlesource.com/38575 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
We're going to start using manually-managed spans for GC workbufs, so rename the allocate/free methods and pass in a pointer to the stats to use instead of using the stack stats directly. For #19325. Change-Id: I37df0147ae5a8e1f3cb37d59c8e57a1fcc6f2980 Reviewed-on: https://go-review.googlesource.com/38576 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, manually-managed spans are included in memstats.heap_inuse and memstats.heap_sys, but when we export these stats to the user, we subtract out how much has been allocated for stack spans from both. This works for now because stacks are the only manually-managed spans we have. However, we're about to use manually-managed spans for more things that don't necessarily have obvious stats we can use to adjust the user-presented numbers. Prepare for this by changing the accounting so manually-managed spans don't count toward heap_inuse or heap_sys. This makes these fields align with the fields presented to the user and means we don't have to track more statistics just so we can adjust these statistics. For #19325. Change-Id: I5cb35527fd65587ff23339276ba2c3969e2ad98f Reviewed-on: https://go-review.googlesource.com/38577 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Rick Hudson <rlh@golang.org>
This clarifies that the gcBits type is actually an arena of gcBits and will let us introduce a new gcBits type representing a single mark/alloc bitmap allocated from the arena. For #19325. Change-Id: Idedf76d202d9174a17c61bcca9d5539e042e2445 Reviewed-on: https://go-review.googlesource.com/38579 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
This introduces a new type, *gcBits, to use for alloc/mark bitmap allocations instead of *uint8. This type is marked go:notinheap, so uses of it correctly eliminate write barriers. Since we now have a type, this also extracts some common operations to methods both for convenience and to avoid (*uint8) casts at most use sites. For #19325. Change-Id: Id51f734fb2e96b8b7715caa348c8dcd4aef0696a Reviewed-on: https://go-review.googlesource.com/38580 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
Currently the runtime allocates workbufs from persistent memory, which means they can never be freed. Switch to allocating them from manually-managed heap spans. This doesn't free them yet, but it puts us in a position to do so. For #19325. Change-Id: I94b2512a2f2bbbb456cd9347761b9412e80d2da9 Reviewed-on: https://go-review.googlesource.com/38581 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
Currently GC workbufs are in off-heap memory that never gets freed. The memory allocated for these is (roughly) proportional to peak heap size, so applications that have temporary high peaks (such as during initialization) suffer from high RSS usage for the rest of the process' life time.
Unlike various other off-heap runtime structures, these would actually be pretty easy to free. We should do that. I think the only subtlety would be figuring out how many we want to keep, since we probably don't want to free all of the workbufs after every GC cycle.
/cc @RLH
The text was updated successfully, but these errors were encountered: