I believe this is also true for 1.4 so it isn't a problem we are
introducing in 1.5. When to trigger a GC is part of the problem and Austin
is working on that. The work includes dampening functions to control
oscillations but it is not intended to deal with constrained memory GC.

That said I do believe that constrained memory GC is an important part of
the design space and when Go starts to move in that direction I am sure we
will tackle the problem.

On Tue, Mar 3, 2015 at 2:29 AM, Dmitry Vyukov notifications@github.com
wrote:

Currently GC allows heap to grow to 2x of live memory during the previous
GC. This can play badly with spiky memory usage. Consider that in steady
state program has live set X. GC will allow heap to grow to 2X and then
collect it back to X, and so on. Now consider that there is a 1.5X spike in
memory usage. If GC happens after the spike (when live set is again X),
then GC will collect X memory (garbage generated during the spike) and set
heap limit to 2X as before. Now if GC happens to happen during the
spike (when live set is 1.5X), then GC will collect only 0.5X and set heap
limit to 3X.

Basically bad timing can increase maximum heap (RSS) by up to 2x.

Memory-constrained environments, like browsers, pay a great deal of
attention to this problem. The idea is to set smarter GC threshold when
heap grows/shrinks.

I did not work out a solution. But what I have in mind is: if heap grows,
and especially if the next threshold (next_gc) will be larger than the
current RSS (heap_sys - heap_released), then set next_gc to, say,
heap_inuse * (1 + GOGC/100) * 0.75.
Since heap cannot grow all the time, this throttling is only temporal.

@rsc https://github.com/rsc @RLH https://github.com/RLH @aclements
https://github.com/aclements @randall77 https://github.com/randall77

—
Reply to this email directly or view it on GitHub
#10064.

As Rick mentioned, I think my GC scheduler design should help with this.
I'm going to send out the design doc in the next few days. It doesn't
directly damp the target heap size, but it does damp other things like the
GC trigger point that should indirectly damp the target heap size. Once
it's implemented, we can see how much this helps and if it's not enough we
can integrate some direct damping of the base heap size.
On Mar 3, 2015 8:59 AM, "Richard L. Hudson" notifications@github.com
wrote:

I believe this is also true for 1.4 so it isn't a problem we are
introducing in 1.5. When to trigger a GC is part of the problem and Austin
is working on that. The work includes dampening functions to control
oscillations but it is not intended to deal with constrained memory GC.

That said I do believe that constrained memory GC is an important part of
the design space and when Go starts to move in that direction I am sure we
will tackle the problem.

On Tue, Mar 3, 2015 at 2:29 AM, Dmitry Vyukov notifications@github.com
wrote:

Currently GC allows heap to grow to 2x of live memory during the previous
GC. This can play badly with spiky memory usage. Consider that in steady
state program has live set X. GC will allow heap to grow to 2X and then
collect it back to X, and so on. Now consider that there is a 1.5X spike
in
memory usage. If GC happens after the spike (when live set is again X),
then GC will collect X memory (garbage generated during the spike) and
set
heap limit to 2X as before. Now if GC happens to happen during the
spike (when live set is 1.5X), then GC will collect only 0.5X and set
heap
limit to 3X.

Basically bad timing can increase maximum heap (RSS) by up to 2x.

Memory-constrained environments, like browsers, pay a great deal of
attention to this problem. The idea is to set smarter GC threshold when
heap grows/shrinks.

I did not work out a solution. But what I have in mind is: if heap grows,
and especially if the next threshold (next_gc) will be larger than the
current RSS (heap_sys - heap_released), then set next_gc to, say,
heap_inuse * (1 + GOGC/100) * 0.75.
Since heap cannot grow all the time, this throttling is only temporal.

@rsc https://github.com/rsc @RLH https://github.com/RLH @aclements
https://github.com/aclements @randall77 https://github.com/randall77

—
Reply to this email directly or view it on GitHub
#10064.

—
Reply to this email directly or view it on GitHub
#10064 (comment).

Not for Go 1.5 at least.

@aclements, you happy with Go 1.8's behavior here?