Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: fractional workers can cause 20ms mutator latency #21698

Closed
aclements opened this issue Aug 30, 2017 · 3 comments
Closed

runtime: fractional workers can cause 20ms mutator latency #21698

aclements opened this issue Aug 30, 2017 · 3 comments
Milestone

Comments

@aclements
Copy link
Member

What version of Go are you using (go version)?

1.9 (or tip b039215)

What did you expect to see?

Fractional mark workers should have minimal impact on mutator latency.

What did you see instead?

The fractional mark worker often runs for 20ms at a time, which means other goroutines on the same P can be significantly delayed. (In contrast, dedicated mark workers kick all other goroutines on the P to the global run queue, where they're more likely to get picked up by other Ps.)

We should consider imposing a tight preemption bound of fractional workers (they'd have to voluntarily preempt without sysmon's help). At larger GOMAXPROCS, we may want to simply not use fractional workers.

This problem, and others, was observed in #18534 (see, in particular, #18534 (comment)).

/cc @RLH @tarm

@gopherbot
Copy link

Change https://golang.org/cl/68573 mentions this issue: runtime: preempt fractional worker after reaching utilization goal

@gopherbot
Copy link

Change https://golang.org/cl/68574 mentions this issue: runtime: schedule fractional workers on all Ps

@gopherbot
Copy link

Change https://golang.org/cl/68571 mentions this issue: runtime: use only dedicated mark workers at reasonable GOMAXPRCS

gopherbot pushed a commit that referenced this issue Oct 13, 2017
When GOMAXPROCS is not small, fractional workers don't add much to
throughput, but they do add to the latency of individual goroutines.
In this case, it makes sense to just use dedicated workers, even if we
can't exactly hit the 25% CPU goal with dedicated workers.

This implements this logic by computing the number of dedicated mark
workers that will us closest to the 25% target. We only fall back to
fractional workers if that would be more than 30% off of the target
(less than 17.5% or more than 32.5%, which in practice happens for
GOMAXPROCS <= 3 and GOMAXPROCS == 6).

Updates #21698.

Change-Id: I484063adeeaa1190200e4ef210193a20e635d552
Reviewed-on: https://go-review.googlesource.com/68571
Reviewed-by: Rick Hudson <rlh@golang.org>
gopherbot pushed a commit that referenced this issue Oct 13, 2017
Currently only a single P can run a fractional mark worker at a time.
This doesn't let us spread out the load, so it gets concentrated on
whatever unlucky P picks up the token to run a fractional worker. This
can significantly delay goroutines on that P.

This commit changes this scheduling rule so each P separately
schedules fractional workers. This can significantly reduce the load
on any individual P and allows workers to self-preempt earlier. It
does have the downside that it's possible for all Ps to be in
fractional workers simultaneously (an effect STW).

Updates #21698.

Change-Id: Ia1e300c422043fa62bb4e3dd23c6232d81e4419c
Reviewed-on: https://go-review.googlesource.com/68574
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
@golang golang locked and limited conversation to collaborators Oct 13, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants