[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Proposal: block-based vector allocator

From: Stefan Monnier
Subject: Re: Proposal: block-based vector allocator
Date: Thu, 08 Dec 2011 14:58:04 -0500
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.91 (gnu/linux)

>> Someone could try to change the allocator so those blocks are exactly
>> 1-page in size and get allocated via mmap:
> I did this even before I started with block-based vector allocator.

How much did it help?

>> - using mmap will ensure that we factor out malloc from the equation.
>> - using page-size (presumably 4KB) blocks rather than the current 16KB
> Current sizes are 1K except string data, which is 8K.

For cons cells and floats, we call malloc for 16KB blocks at a time
(which we then cut into 1024 blocks).  This is because we need the 1024
blocks to be aligned on a 1024 boundary (so when we use malloc instead
of posix_memalign, we waste upto 1023 (well 1016, really) bytes of those

> My favorite byte-force-recompile benchmark usually consumes ~89M. With my
> vector allocator it shrinks to 78M.

And you said that using the non-conservative stack scanning resulted in
similar results, so all that extra memory is occupied by mem_nodes (or
fragments that used to hole mem_nodes ;-).

> But for (definitely pathological) compile buffer case, results are
> very unexpected (I tried with 100000-lines buffer).  Fresh instance
> is 18M.  With compile buffer, it grows to 142M and shrinks to 141M
> after killing *compilation* and garbage-collect.  When mmap() is enabled,
> it grows to 138M and shrinks back to 21M (twenty one) after kill & GC.

Hmm.. cool.  So some of the responsability for this test to behave so
poorly might be in malloc rather than in alloc.c.  Good news.
Oh, yes, now I remember:

      /* Prevent mmap'ing the chunk.  Lisp data may not be mmap'ed
         because mapped region contents are not preserved in
         a dumped Emacs.  */
      mallopt (M_MMAP_MAX, 0);

So maybe malloc behaves poorly simply because we don't let it
behave better.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]