code review 6846061: make.bash: raise ulimit -d to its hard limit to fix Net...

make.bash: raise ulimit -d to its hard limit to fix NetBSD/amd64 build

[minux1, 2012-11-15 19:02:06]

Hello golang-dev@googlegroups.com (cc: golang-dev@googlegroups.com),

I'd like you to review this change to
https://code.google.com/p/go/

[iant, 2012-11-15 19:11:56]

LGTM

https://codereview.appspot.com/6846061/diff/7001/src/make.bash
File src/make.bash (right):

https://codereview.appspot.com/6846061/diff/7001/src/make.bash#newcode119
src/make.bash:119: # On OS X ulimit -S -n rejects 'unlimited'.
s/-n/-d/ I assume

[minux1, 2012-11-15 19:29:30]

i'm still not sure this is the solution for the problem
as this change only fixes the builder. however, i
cannot think of any better ways (limiting the arena to
64GiB doesn't look good to me).

all netbsd/amd64 go user need to raise -d ulimit
for *all* go programs, and this makes me a little
uncomfortable.

should i file an issue for this and also cite that issue
in the comment so that we can come back to it later?

btw, i do have an alternative solution, that is, dynamically
allocate (entire or part of) the mheap structure after we
raise the ulimit. this will have much larger impact so i'm
open to opinions about this.

[iant2, 2012-11-15 20:06:27]

On Thu, Nov 15, 2012 at 11:29 AM, minux <minux.ma@gmail.com> wrote:
>
> all netbsd/amd64 go user need to raise -d ulimit
> for *all* go programs, and this makes me a little
> uncomfortable.

Ah, I didn't understand that.

> btw, i do have an alternative solution, that is, dynamically
> allocate (entire or part of) the mheap structure after we
> raise the ulimit. this will have much larger impact so i'm
> open to opinions about this.

Can we raise the ulimit in runtime·osinit in runtime/thread_netbsd.c?

Ian

[minux1, 2012-11-17 19:57:24]

On Fri, Nov 16, 2012 at 4:06 AM, Ian Lance Taylor <iant@google.com> wrote:

> On Thu, Nov 15, 2012 at 11:29 AM, minux <minux.ma@gmail.com> wrote:
> >
> > all netbsd/amd64 go user need to raise -d ulimit
> > for *all* go programs, and this makes me a little
> > uncomfortable.
>
> Ah, I didn't understand that.
>
the relevant problem description:
https://groups.google.com/group/golang-dev/browse_thread/thread/48598af2cc4d5...

also, the problem is affecting windows/amd64 builder as well,
see https://code.google.com/p/go/issues/detail?id=4379.

>
> > btw, i do have an alternative solution, that is, dynamically
> > allocate (entire or part of) the mheap structure after we
> > raise the ulimit. this will have much larger impact so i'm
> > open to opinions about this.
>
> Can we raise the ulimit in runtime·osinit in runtime/thread_netbsd.c?
>
the problem is that with a data segment slight larger than 256MiB (the
default
soft limit on netbsd/amd64 is 256MiB), exec(2) simply rejects our binary, so
there is no easy workaround like this.

[iant2, 2012-11-18 07:55:41]

On Sat, Nov 17, 2012 at 11:57 AM, minux <minux.ma@gmail.com> wrote:
>
>
> On Fri, Nov 16, 2012 at 4:06 AM, Ian Lance Taylor <iant@google.com> wrote:
>>
>> On Thu, Nov 15, 2012 at 11:29 AM, minux <minux.ma@gmail.com> wrote:
>> >
>> > all netbsd/amd64 go user need to raise -d ulimit
>> > for *all* go programs, and this makes me a little
>> > uncomfortable.
>>
>> Ah, I didn't understand that.
>
> the relevant problem description:
>
https://groups.google.com/group/golang-dev/browse_thread/thread/48598af2cc4d5...
>
> also, the problem is affecting windows/amd64 builder as well,
> see https://code.google.com/p/go/issues/detail?id=4379.
>>
>>
>> > btw, i do have an alternative solution, that is, dynamically
>> > allocate (entire or part of) the mheap structure after we
>> > raise the ulimit. this will have much larger impact so i'm
>> > open to opinions about this.
>>
>> Can we raise the ulimit in runtime·osinit in runtime/thread_netbsd.c?
>
> the problem is that with a data segment slight larger than 256MiB (the
> default
> soft limit on netbsd/amd64 is 256MiB), exec(2) simply rejects our binary, so
> there is no easy workaround like this.

Oh yeah.  Sorry for being slow.

Perhaps dynamically allocating the map field of mheap, if that
suffices, is the best approach.  It sounds like any other approach
will be quite painful for NetBSD users.

Ian