Skip to content

runtime: do not rely on a single bitmap for all Go heap? #9313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
minux opened this issue Dec 14, 2014 · 8 comments
Closed

runtime: do not rely on a single bitmap for all Go heap? #9313

minux opened this issue Dec 14, 2014 · 8 comments

Comments

@minux
Copy link
Member

minux commented Dec 14, 2014

See also #9311 for Android L and #7284 for windows/386. This might also affect
the darwin/arm port, but so far my testing hasn't revealed anything (I think darwin/arm
kernel disallows memory overcommit, so it might poses problems for devices with
less than 1GB of physical memory)

For 32-bit ports, we probably want to support a scheme where there could be separate
bitmap for each large chunk of VM reservation.

Thoughts?

@randall77
Copy link
Contributor

Isn't there a difference between reserving virtual address and committing physical memory to it? We only need the former, not the latter.

@minux
Copy link
Member Author

minux commented Dec 14, 2014

My use of "over-commit" to describe the darwin/arm problem is incorrect. To
rephrase the
problem: the kernel simply doesn't allow you to reserve more VM space than
the amount
of physical memory you have.

Anyway, the ios problem aside, I think the problem is not related to
physical memory, but
related to virtual memory space fragmentation. We might not be able to
reserve a single
768MB space, but perhaps we can reserve three 256MB space, just not
contiguous.

In general, I think supporting multiple bitmaps is very essential for
32-bit systems.

@RLH
Copy link
Contributor

RLH commented Dec 15, 2014

The problem is that we have two runtimes, both of which believe they can control the virtual address space. If the available virtual addresses are contiguous and ART reserves the first n available virtual addresses then the remainder should be contiguous and Go can just grab whatever is left. Does ART grab non-contiguous virtual addresses for some reason?

@crawshaw
Copy link
Member

ART reserves several different address regions: https://android.googlesource.com/platform/art/+/master/runtime/gc/heap.h#797

I don't know enough about ART to say why it does what it does, but they would like to (and believe they can) reduce their reservations to make life easier for native sharing the process. Just not soon.

Empirically, at least one user has been able to reserve a 512MB block followed by several 256MB blocks, but cannot reserve two 512MB blocks.

I will try to do something minimal for #9311 and leave the discussion of multiple (maybe just two?) bitmaps on this issue.

@crawshaw
Copy link
Member

On darwin/arm mmap: Carmack describes being able to mmap 700 MB on the iphone 3GS, which only had 256 MB of physical memory: http://www.bethblog.com/2010/10/29/john-carmack-discusses-rage-on-iphoneipadipod-touch/

The contiguous space limit on iOS is surprisingly low, but their virtual memory works, and I'm hoping the smaller initial arena fallback I just committed will get us started on iOS.

@rsc
Copy link
Contributor

rsc commented Apr 10, 2015

Let's just muddle along until we can stop worrying about 32-bit Android, iOS, or Windows.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@bradfitz
Copy link
Contributor

bradfitz commented Feb 2, 2017

iOS stopped allowing new 32-bit apps shortly after @rsc's last comment.

And as of ~yesterday, the new iOS beta says 32-bit apps won't even run on iOS anymore.

So, Android and Windows.

@eliasnaur, is this a problem in practice for you?

@aclements, you want to keep this open?

@aclements
Copy link
Member

Since Go 1.7 we've been able to accept allocations anywhere in the 32-bit address space without the need for multiple bitmaps (actually, it was @eliasnaur who did that work), so think we can just close this.

Let me know if I've misunderstood the issue and we can reopen.

@golang golang locked and limited conversation to collaborators Feb 2, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

8 participants