Make bumper allocator thread-safe

[Mic92]

We are currently using https://en.cppreference.com/w/cpp/memory/monotonic_buffer_resource.html which is not thread-safe.
For the multi-threaded evaluation we to create a bumper allocator that is thread-safe.

Noted in #14106 (comment)

[Mic92]

Alternativily, can we make the bumper allocator thread_local?

[xokdvium]

Alternativily, can we make the bumper allocator thread_local?

Doesn't seem like a good design decision imo. The ultimate goal is to have a unified 32-bit address space for various types of entities anyway and that means the contiguous array/vector would be a shared resource.

Maybe we could preallocate a big chunk of virtual memory (like 1TB on 64 bit) with mmap. That would give us a contiguous virtual address space that would alleviate the need for thread-safe reallocation / chunk allocation - only an atomic index/pointer bump. Does that sound good? @roberth mentioned that GHC does this as well for its heap.

[Radvendii]

I'm not sure how portable that is. Does GHC do it always, or only when the system allows it?

I suppose we could say "if you don't allow giant pre-allocations you can't have multithreaded evaluation"

[Mic92]

Alternativily, can we make the bumper allocator thread_local?

Doesn't seem like a good design decision imo. The ultimate goal is to have a unified 32-bit address space for various types of entities anyway and that means the contiguous array/vector would be a shared resource.

I don't think this contradicts. These monotonic allocator get internal chunks from an upstream source. This could be a thread-safe malloc from some 4GB range.

[Mic92]

Maybe we could preallocate a big chunk of virtual memory (like 1TB on 64 bit) with mmap. That would give us a contiguous virtual address space that would alleviate the need for thread-safe reallocation / chunk allocation - only an atomic index/pointer bump. Does that sound good? @roberth mentioned that GHC does this as well for its heap.

I actually tested a bumper allocator using atomic index operations and it's significantly slower than normal malloc (which has some sort of local arenas per thread). Thread-local bumper allocator where much faster with increasing threads than both normal malloc and what I came up with.

[Ericson2314]

I do think that the expr address spaces can be per-file, and parsing is still single threaded. The only question then is what to do with the main value heap.

My long term thinking is what once we have compaction, temporary partitions on the address space are not so bad. If/when one evaluation worker hits the limit on its partition, GC is triggered, and we defrag and can repartition. Only when the address space is low / such redivide and quotas-hits are happening to frequently, might we switch to single bump allocator with synchronization (which will be slower, but not defragmenting).