code review 5725045: runtime: try extending arena size in 32-bit allocator.

runtime: try extending arena size in 32-bit allocator.

If it didn't reach the limit, we can try extending the arena
before resorting to random memory mappings and praying for the
kernel to be kind.

Fixes issue 3173.

[remyoudompheng, 2012-03-06 20:30:19]

Hello golang-dev@googlegroups.com (cc: golang-dev@googlegroups.com),

I'd like you to review this change to
https://go.googlecode.com/hg/

[remyoudompheng, 2012-03-06 20:31:03]

This is an attempt for issue 3173. I'd like to know if anyone has an idea about
it.

[rsc, 2012-03-06 20:33:44]

I think (having not looked) that the bug is that the fallback
code is not running at all.  Have you checked?  Is it running
and failing?

Russ

[remyoudompheng, 2012-03-06 20:36:34]

Do you mean the call to runtime·SysAlloc? It returns something outside the
possible arena (i don't really expect that it would comply anyway), as I see in
the "debug" version of the error message:

runtime: memory allocated by OS (0xd75c4000) not in usable range
[0x18500000,0x98500000)

[rsc, 2012-03-07 02:44:31]

On Tue, Mar 6, 2012 at 15:36,  <remyoudompheng@gmail.com> wrote:
> Do you mean the call to runtime·SysAlloc? It returns something outside
> the possible arena (i don't really expect that it would comply anyway),
> as I see in the "debug" version of the error message:
>
> runtime: memory allocated by OS (0xd75c4000) not in usable range
> [0x18500000,0x98500000)

Aha!

Russ

[rsc1, 2012-03-07 02:45:11]

LGTM

[remyoudompheng, 2012-03-07 07:13:55]

Hello rsc@golang.org (cc: golang-dev@googlegroups.com),

Please take another look.

[remyoudompheng, 2012-03-07 07:14:27]

http://codereview.appspot.com/5725045/diff/5001/src/pkg/runtime/malloc.goc
File src/pkg/runtime/malloc.goc (right):

http://codereview.appspot.com/5725045/diff/5001/src/pkg/runtime/malloc.goc#ne...
src/pkg/runtime/malloc.goc:378: new_end = (byte*)((((uintptr)new_end >> 28) + 1)
<< 28);
oh, I'm rounding address instead of arena size.

[rsc1, 2012-03-07 07:26:40]

LGTM

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc
File src/pkg/runtime/malloc.goc (right):

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc#ne...
src/pkg/runtime/malloc.goc:378: uintptr needed = (uintptr)h->arena_used + n -
(uintptr)h->arena_end;
Please make this uintptr needed; and move the initialization down after the
blank line.

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc#ne...
src/pkg/runtime/malloc.goc:379: // Round wanted arena size to a multiple of
256MB.
256<<20 is a more common way to write 256 MB.

[remyoudompheng, 2012-03-07 07:35:32]

Hello rsc@golang.org, rsc@google.com (cc: golang-dev@googlegroups.com),

Please take another look.

[remyoudompheng, 2012-03-07 07:35:55]

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc
File src/pkg/runtime/malloc.goc (right):

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc#ne...
src/pkg/runtime/malloc.goc:378: uintptr needed = (uintptr)h->arena_used + n -
(uintptr)h->arena_end;
On 2012/03/07 07:26:40, rsc1 wrote:
> Please make this uintptr needed; and move the initialization down after the
> blank line.

Done.

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc#ne...
src/pkg/runtime/malloc.goc:379: // Round wanted arena size to a multiple of
256MB.
On 2012/03/07 07:26:40, rsc1 wrote:
> 256<<20 is a more common way to write 256 MB.

Done.

http://codereview.appspot.com/5725045/diff/8002/src/pkg/runtime/malloc.goc#ne...
src/pkg/runtime/malloc.goc:379: // Round wanted arena size to a multiple of
256MB.
On 2012/03/07 07:26:40, rsc1 wrote:
> 256<<20 is a more common way to write 256 MB.

Done.

[rsc, 2012-03-07 19:14:03]

LGTM

[rsc, 2012-03-07 19:21:47]

*** Submitted as http://code.google.com/p/go/source/detail?r=3d7bfd2a407a ***

runtime: try extending arena size in 32-bit allocator.

If it didn't reach the limit, we can try extending the arena
before resorting to random memory mappings and praying for the
kernel to be kind.

Fixes issue 3173.

R=rsc, rsc
CC=golang-dev
http://codereview.appspot.com/5725045

Committer: Russ Cox <rsc@golang.org>