cc @mknyszek

Thanks for the detailed experiments and data.

There's a more direct way to force huge pages that I'd been considering: we already have sysHugePage in the runtime which just just madvise(MADV_HUGEPAGE). I appreciate your experiments there, it shows that this is worthwhile to make work. (This should probably be better than zeroing, because in theory the OS is already giving us zeroed pages so the zeroing is redundant? Unless I'm missing something.)

Also I've got patches on the prototype out to:

1. Don't actually unmap the region, just re-map it as PROT_NONE. That's not actually safe since something else could map there.
2. Make the 64 MiB chunk eligible for reuse once the GC finds no pointers into the region.

Your experiments on partial reuse are also great. I feel like that that point we should just use smaller arenas instead. My patch for (2) causes heapArena metadata to retain a 1:1 mapping with address space (as opposed to now, where the heapArena might be used for something else), so it'll be really quite feasible to handle smaller chunks, or partial chunks, in general.

Hi @mknyszek

There's a more direct way to force huge pages that I'd been considering: we already have sysHugePage in the runtime which just just madvise(MADV_HUGEPAGE).

FWIW, I did try that as well in a couple of different spots, and it did not seem to make a difference, but perhaps I did not call it in the right spot or made some other mistake.

Interesting. THP has always been a little specific about its requirements. When I get back to this (arenas in general, I mean, got a bunch of other stuff on my plate right now) I'll experiment with that some more.

Also, it looked like sysHugePage might already be called in the original CL 387975 via reflect_unsafe_newUserArenaChunk -> allocSpan -> sysUsed -> sysHugePage, but not 100% confident in that.

@thepudds Based on your suggestions I implemented https://go.dev/cl/423361. I think I included all your suggestions, though rather than changing the reuse policy I just made arenas 8 MiB in size to begin with. Checking against the same benchmark, I got roughly the same performance difference you got, so yay reproducibility!

Thanks again for the time you took to analyze this more closely. :)

@mknyszek, that's great to hear, and glad it was helpful!

FWIW, I think there are likely some implications from these improvements... I had a few related comments in #51317, including this snippet:

My guess (which might be wrong! ;-) based on what I saw in #51667 is that if arenas are adopted and there is a full implementation, it might be plausible to use 100s or perhaps even 1000s of them concurrently rather than "few to 10s" [from original proposal], though of course it would likely depend on how much RAM you have, how much benefit you are getting, and especially at the upper end it might come down to CPU vs. RAM tradeoffs.

If it ends up being 100s or 1000s concurrently, that would of course have some implications for how widely they are used, how viral they are in terms of APIs, and so on.

One API implication of wanting to use more arenas concurrently is that there might be some benefit to allowing a user to control or hint the batching size or chunk size. That said, that might not be worthwhile if the chunk size and/or batch size is "small enough" or perhaps auto-tuned. Also, the API could possibly allow getting a snapshot of how many allocations have been done by an arena instance. That could be helpful for things like testing and monitoring, and could allow a user to more conveniently decide to re-use an arena instance or not, although a user could in theory track themselves.

One API implication of wanting to use more arenas concurrently is that there might be some benefit to allowing a user to control or hint the batching size or chunk size.

This is more possible with https://go.dev/cl/423361 since the arena chunk sizes are now decoupled from the heap arena size. You can't have a chunk size larger than a heap arena at the moment, but any power-of-two smaller works.

As for having lots of different chunk sizes, that becomes straightforward if the chunks are just taken from the page allocator, since differently-sized chunks can be returned to the page heap instead of sitting on a list. It wouldn't be too hard to do, I don't think. Something worth considering in the future perhaps!

Hi, thanks so much for working on the arena GOEXPERIMENT! I took it for a test drive today, and found it very usable. However, I quickly ran into the limitations discussed above around many concurrently live arenas.

I'm interested to see if arenas could be applicable for CockroachDB's use cases. In particular, during the query planning phase, CockroachDB performs a large number of allocations that have identical or very similar lifetimes. It would be impactful to be able to arena allocate these allocations.

However, CockroachDB typically hosts a large number of concurrent SQL sessions - in the 1000s. Because of this, the use case I'm describing would really need configurable arena chunk sizes, like you've mentioned above.

So, +1 to that idea!

The other thing that I wanted to mention was that it would be impactful to be able to inspect an arena to understand the amount of memory reserved for it. Without that, it would be tough for the CockroachDB use case I mentioned to use arenas as implemented, because CockroachDB needs to account for memory that it allocates to ensure that when running out of memory the server can react appropriately (e.g. queuing or rejecting incoming queries, and causing queries to spill disk).