[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Proposal: block-based vector allocator
From: |
Stefan Monnier |
Subject: |
Re: Proposal: block-based vector allocator |
Date: |
Thu, 08 Dec 2011 09:10:54 -0500 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.0.91 (gnu/linux) |
>> I don't think you're going to solve the memory use issues by improving
>> vector allocation.
> Initially I considered the possibility to use mmap()ed areas to store Lisp
> data for normal (already dumped) Emacs. Among others, it requires
> 1) suitable tagging scheme and 2) ability to store all Lisp data within
> multiple-of-page-size blocks. That was the main motivation for working on
> block-based vector allocator.
Emacs already allocates all cons, float, string headers,
markers/overlays, interval nodes, and symbols (i.e. everything other
than vectors) within (more or less) multiple-of-page-size blocks.
These are allocated via `malloc' rather than `mmap', but I don't think
our "memory issues" are due to malloc not returning the corresponding
pages to the OS, but rather due to the fact that after a phase of very
high allocation followed by a massive deallocation, many of these blocks
end up mostly (but not fully) empty.
Now, I don't know for a fact that this is the case, it's just a hunch.
Someone could try to change the allocator so those blocks are exactly
1-page in size and get allocated via mmap:
- using mmap will ensure that we factor out malloc from the equation.
- using page-size (presumably 4KB) blocks rather than the current 16KB
reduces the granularity so there's more probability that a block
contains no live objects.
Then re-run some problematic case (e.g. the million-lines compile buffer)
to see if this helps. It really should help, tho I don't hold much hope
that it will make a significant difference (e.g. more than 30%
difference would be significant).
>> If you want to keep working on it, fine -- every little bit helps, and
>> you might succeed. But I don't think it should be a priority for
>> Emacs, and you are going to need to show more gains to make it worth a
>> somewhat more complex code base.
> Sorry, but did you mean "don't show half-done work to a fool"? If yes,
> then I should agree.
It's not a priority for Emacs to improve the CPU and memory performance
in general. But I do find it important to make Elisp programming
easier, which is why I'm interested in a more complex vector allocation:
defstruct is currently pathetically inefficient.
Stefan
- Re: Proposal: block-based vector allocator, (continued)
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/06
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/07
- Re: Proposal: block-based vector allocator, Carsten Mattner, 2011/12/07
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/07
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/07
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/07
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/08
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/08
- Re: Proposal: block-based vector allocator, Stephen J. Turnbull, 2011/12/07
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/07
- Re: Proposal: block-based vector allocator,
Stefan Monnier <=
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/08
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/08
- Re: Proposal: block-based vector allocator, Eli Zaretskii, 2011/12/09
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/09
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/09
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/09
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/09
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/11
- Re: Proposal: block-based vector allocator, Dmitry Antipov, 2011/12/11
- Re: Proposal: block-based vector allocator, Stefan Monnier, 2011/12/12