code review 5309070: runtime: lock the main goroutine to the main OS thread

runtime: lock the main goroutine to the main OS thread during init

We only guarantee that the main goroutine runs on the
main OS thread for initialization.  Programs that wish to
preserve that property for main.main can call runtime.LockOSThread.
This is what programs used to do before we unleashed
goroutines during init, so it is both a simple fix and keeps
existing programs working.

[rsc, 2011-10-27 02:41:21]

Hello iant, r (cc: golang-dev@googlegroups.com),

I'd like you to review this change to
https://go.googlecode.com/hg/

[r, 2011-10-27 03:12:56]

I pre-emptively claim this as too expensive, and surprisingly and
unpredictably to boot.  Every little demo of concurrency has the
chance to become much too slow.

This is distorting the language to fix a particular, rare problem in a
particular, specific setup. There must be a less invasive solution.
What exactly needs to run on the main thread, and how can we make that
happen?

-rob

[iant2, 2011-10-27 04:04:27]

"Rob 'Commander' Pike" <r@golang.org> writes:

> I pre-emptively claim this as too expensive, and surprisingly and
> unpredictably to boot.  Every little demo of concurrency has the
> chance to become much too slow.
>
> This is distorting the language to fix a particular, rare problem in a
> particular, specific setup. There must be a less invasive solution.
> What exactly needs to run on the main thread, and how can we make that
> happen?

It really does not sound like the right approach to me, either.  But I
don't fully understand the problem.

Ian

[rsc, 2011-10-27 04:27:22]

On Wed, Oct 26, 2011 at 20:12, Rob 'Commander' Pike <r@golang.org> wrote:
> I pre-emptively claim this as too expensive, and surprisingly and
> unpredictably to boot.  Every little demo of concurrency has the
> chance to become much too slow.

Only demos with 2 goroutines.  Demos with 100,000 goroutines are fine,
because 99,999 of them will be in a single thread.  And it is far from clear
that this microbenchmarks have real relevance.  But I take your point.

> This is distorting the language to fix a particular, rare problem in a
> particular, specific setup. There must be a less invasive solution.
> What exactly needs to run on the main thread, and how can we make that
> happen?

It is distorting the implementation, not the language.
I was very careful not to say anything about this in the spec:
it truly is an implementation detail.

It is possible, as the comment says, that we can restrict
the locking to OS X programs that use cgo, since that is
the case where we have a demonstrated need.  A middle
ground would be any program that uses cgo (runtime can
tell, even at that early stage).

Russ

[rsc, 2011-10-27 04:31:03]

On Wed, Oct 26, 2011 at 21:04, Ian Lance Taylor <iant@google.com> wrote:
> It really does not sound like the right approach to me, either.  But I
> don't fully understand the problem.

The problem is that OS X requires that you call certain API functions
from the 'main thread', meaning the thread where program execution
begins.  This change would make the main thread be guaranteed to
coincide with the main goroutine, the one that runs initialization and
main.main.

If you allow other goroutines to run in the main thread, one may block
that thread for an indefinite amount of time, making it impossible to
run an API call in the main goroutine.  This CL avoids that problem by
reserving the main thread for the main goroutine.

Russ

[r, 2011-10-27 04:36:06]

But doesn't essentially every program use cgo now, because of networking?

-rob

[rsc, 2011-10-27 04:51:33]

On Wed, Oct 26, 2011 at 21:36, Rob 'Commander' Pike <r@golang.org> wrote:
> But doesn't essentially every program use cgo now, because of networking?

I believe that any real Go program is not going to be bottlenecked
by the speed of channel communication between a pair of goroutines.
We decided that was not a case to promote: it's why we removed
the 'channel as iterator' methods from the containers.

The only programs I expect this CL to affect are microbenchmark
channel pingpong programs, and those typically do not import "net",
so the cgo heuristic would be a way to avoid affecting them.

It is a clumsy distinction, though; hence my suggestion that
we apply it to all programs and refine the condition only if, in practice,
it makes a measurable impact on real programs.  I am worried
that too much focus on microbenchmarks is keeping us from
doing things that matter in real programs.

Russ

[dfc, 2011-10-27 05:16:30]

Hi Russ,

How would the 'only if you use CGO' requirement fit with something that is built
with pkg/net? I think the cgo detection would be too easily fooled. 

Cheers

Dave

Sent from my iPhone

On 27/10/2011, at 15:27, Russ Cox <rsc@golang.org> wrote:

> On Wed, Oct 26, 2011 at 20:12, Rob 'Commander' Pike <r@golang.org> wrote:
>> I pre-emptively claim this as too expensive, and surprisingly and
>> unpredictably to boot.  Every little demo of concurrency has the
>> chance to become much too slow.
> 
> Only demos with 2 goroutines.  Demos with 100,000 goroutines are fine,
> because 99,999 of them will be in a single thread.  And it is far from clear
> that this microbenchmarks have real relevance.  But I take your point.
> 
>> This is distorting the language to fix a particular, rare problem in a
>> particular, specific setup. There must be a less invasive solution.
>> What exactly needs to run on the main thread, and how can we make that
>> happen?
> 
> It is distorting the implementation, not the language.
> I was very careful not to say anything about this in the spec:
> it truly is an implementation detail.
> 
> It is possible, as the comment says, that we can restrict
> the locking to OS X programs that use cgo, since that is
> the case where we have a demonstrated need.  A middle
> ground would be any program that uses cgo (runtime can
> tell, even at that early stage).
> 
> Russ

[iant2, 2011-10-27 05:37:47]

Russ Cox <rsc@golang.org> writes:

> On Wed, Oct 26, 2011 at 21:04, Ian Lance Taylor <iant@google.com> wrote:
>> It really does not sound like the right approach to me, either.  But I
>> don't fully understand the problem.
>
> The problem is that OS X requires that you call certain API functions
> from the 'main thread', meaning the thread where program execution
> begins.  This change would make the main thread be guaranteed to
> coincide with the main goroutine, the one that runs initialization and
> main.main.
>
> If you allow other goroutines to run in the main thread, one may block
> that thread for an indefinite amount of time, making it impossible to
> run an API call in the main goroutine.  This CL avoids that problem by
> reserving the main thread for the main goroutine.

Suppose we just don't use the main thread on OS X, except for these
magic operations.

Or, on OS X, let's lock the main goroutine to the main thread at program
startup, and then release it before starting main, unless the program
calls runtime.LockOSThread itself in some init routine.  Because we have
locked the main goroutine, all init routines will run on the main
thread.

Ian

[rsc, 2011-10-27 05:40:57]

On Wed, Oct 26, 2011 at 22:16, Dave Cheney <dave@cheney.net> wrote:
> How would the 'only if you use CGO' requirement fit with
> something that is built with pkg/net? I think the cgo detection
> would be too easily fooled.

The question is what the cost of being fooled is.

The only measurable cost I am aware of is that
pingponging between two goroutines would be
noticeably slower when one goroutine is the main
goroutine.  As far as I am aware that only happens
in synthetic microbenchmarks.

Russ

[rsc, 2011-10-27 06:02:07]

On Wed, Oct 26, 2011 at 22:37, Ian Lance Taylor <iant@google.com> wrote:
> Or, on OS X, let's lock the main goroutine to the main thread at program
> startup, and then release it before starting main, unless the program
> calls runtime.LockOSThread itself in some init routine.  Because we have
> locked the main goroutine, all init routines will run on the main
> thread.

How about doing that on all systems: during initialization the main
goroutine is locked to the main thread.  Again, not a spec, just an
implementation convention.

Russ

[r2, 2011-10-27 06:33:09]

> How about doing that on all systems: during initialization the main
> goroutine is locked to the main thread.  Again, not a spec, just an
> implementation convention.

i could live with that, but i'm pretty unhappy about it. i fear a
slippery slope where every stupid OS decision (and they're starting to
come fast an furious) puts pressure on the implementation and, in
effect, the language.

it may be the best we can do, except for OS X only. is there a
compelling reason to do it everywhere?

-rob

[dvyukov, 2011-10-27 07:46:57]

I have a bad feeling about locking main goroutine to main thread either.
However, it's merely intuition.

Another possible solution is to provide runtime.LockOnMainThread(), then a
program can do:
func main() {
  runtime.LockOnMainThread()
  ...
}
or even 
func main() {
  go eventHandler()
  ...
}
func eventHandler() {
  runtime.LockOnMainThread()
  ...
}
However, it's also amenable to the problem pointed by Russ - third party package
starts a goroutine during init that initiates an infinite system call on the
main thread. Then, main thread can be reserved for execution of the goroutine
that calls LockOnMainThread() (parked otherwise). It is similar to proposed
RunOnMainThread(f func()), but not callback based.


Another solution is to lock main thread before init and unblock before main.
Then a user program can carry on with locking in it's init. It requires
recursion support in LockOSThread() (not necessary officially declared). The
sequence of events is:

LockOSThread() // locks main goroutine to main thread

// packages inits run

func init() { // main init
  LockOSThread() // bumps lock count
}

UnlockOSThread() // decrements lock count

func main() {
  // locked to main thread
  ...
}

It looks like the cheapest and most handy for user solution, was it considered?

[rsc, 2011-10-27 07:59:16]

On Wed, Oct 26, 2011 at 23:33, Rob 'Commander' Pike <r@google.com> wrote:
>> How about doing that on all systems: during initialization the main
>> goroutine is locked to the main thread.  Again, not a spec, just an
>> implementation convention.
>
> i could live with that, but i'm pretty unhappy about it. i fear a
> slippery slope where every stupid OS decision (and they're starting to
> come fast an furious) puts pressure on the implementation and, in
> effect, the language.
>
> it may be the best we can do, except for OS X only. is there a
> compelling reason to do it everywhere?

the reason to do it everywhere is just to reduce variation among
implementations.  it helps to reduce the ways in which OS X
(and more often Windows) are exceptions as much as possible.

i don't see this as a language issue.  it's something we can try
in the implementation, and if it doesn't work well we can try
something else.  locking the thread just during init is easy to
explain, sufficient for the problems people have run into, and
has zero performance impact once you get to main.

russ

[dvyukov, 2011-10-27 08:12:53]

On 2011/10/27 07:59:16, rsc wrote:
> On Wed, Oct 26, 2011 at 23:33, Rob 'Commander' Pike <mailto:r@google.com>
wrote:
> >> How about doing that on all systems: during initialization the main
> >> goroutine is locked to the main thread. &nbsp;Again, not a spec, just an
> >> implementation convention.
> >
> > i could live with that, but i'm pretty unhappy about it. i fear a
> > slippery slope where every stupid OS decision (and they're starting to
> > come fast an furious) puts pressure on the implementation and, in
> > effect, the language.
> >
> > it may be the best we can do, except for OS X only. is there a
> > compelling reason to do it everywhere?
> 
> the reason to do it everywhere is just to reduce variation among
> implementations.  it helps to reduce the ways in which OS X
> (and more often Windows) are exceptions as much as possible.
> 
> i don't see this as a language issue.  it's something we can try
> in the implementation, and if it doesn't work well we can try
> something else.  locking the thread just during init is easy to
> explain, sufficient for the problems people have run into, and
> has zero performance impact once you get to main.

Looks like the best solution in my view.

[rsc, 2011-10-27 14:53:51]

PTAL

I have updated the code to force init (only) onto the main thread.
After init is done, unless an init function called runtime.LockOSThread,
the forced association disappears.  Programs that do not use
runtime.LockOSThread will be unaffected except during init.

Programs that do care can call runtime.LockOSThread in an init
function and then assume that the main goroutine will be locked
to the main thread even during main.main.  Coincidentally, this
is exactly what programs that cared were already doing before
we allowed goroutines during init.  So those programs will just
continue to work.

Russ

[r, 2011-10-27 15:47:26]

LGTM at least as an experiment.

[iant, 2011-10-27 16:19:50]

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c
File src/pkg/runtime/proc.c (right):

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c#newco...
src/pkg/runtime/proc.c:1564: if(m == &runtime·m0 && runtime·sched.init) {
The logic here does not look right.  It seems that this condition will be true
when this is called from runtime·main.  The effect will be that lockmain is
always set to true.

[dvyukov, 2011-10-27 16:21:28]

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c
File src/pkg/runtime/proc.c (right):

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c#newco...
src/pkg/runtime/proc.c:221: 
Delete empty line.

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c#newco...
src/pkg/runtime/proc.c:228: runtime·sched.init = true;
Where do you reset it to false?

[dvyukov, 2011-10-27 16:23:36]

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c
File src/pkg/runtime/proc.c (right):

http://codereview.appspot.com/5309070/diff/14001/src/pkg/runtime/proc.c#newco...
src/pkg/runtime/proc.c:229: runtime·LockOSThread();
Is m already set to m0 here? If yes, LockOSThread will merely set lockmain. I
think the above 2 lines better be swapped.

[rsc, 2011-10-27 19:30:37]

PTAL

I should not write code first thing in the morning.
Yay code reviews.

[iant, 2011-10-27 19:44:29]

LGTM

[rsc, 2011-10-28 01:04:18]

*** Submitted as http://code.google.com/p/go/source/detail?r=d51031a1d932 ***

runtime: lock the main goroutine to the main OS thread during init

We only guarantee that the main goroutine runs on the
main OS thread for initialization.  Programs that wish to
preserve that property for main.main can call runtime.LockOSThread.
This is what programs used to do before we unleashed
goroutines during init, so it is both a simple fix and keeps
existing programs working.

R=iant, r, dave, dvyukov
CC=golang-dev
http://codereview.appspot.com/5309070