Re: Proposal: block-based vector allocator

[Dmitry Antipov]

On 12/08/2011 11:58 PM, Stefan Monnier wrote:

> How much did it help?

Not too much because it makes no sense without block-based vector
allocation - huge amounts of small vectors and corresponding mem_nodes
allocated from brk()ed heap devastates everything. And it was a bit
slower, most probably because explicit mmmap() implies zeroing
performed by OS.

> And you said that using the non-conservative stack scanning resulted in
> similar results, so all that extra memory is occupied by mem_nodes (or
> fragments that used to hole mem_nodes ;-).

Yes, one mem_node per each vector results in substantial overhead and
fragmentation. _Any_ block-based vector allocator should reduce the number
of mem_nodes, so saving space - but rounding allocator will consume some
of this space back for it's own internal fragmentation. That's why
I proposed the scheme with tightest possible fit instead of 'standard
malloc algorithm'.

Even for non-conservative stack scanning with eliminated mem_nodes overhead,
I'm pretty sure that my proposal works at least not worse, taking both
CPU and memory usage into account; rounding vector allocator will consume
more memory, but may have near to negligible advantage in CPU usage because
it has no splitting/coalescing overhead.

Dmitry