Re: When should ralloc.c be used?

[Daniel Colascione]

On 10/24/2016 08:40 AM, Eli Zaretskii wrote:
> > From: Stefan Monnier <address@hidden>
> > Cc: address@hidden,  address@hidden
> > Date: Mon, 24 Oct 2016 10:37:19 -0400
> >
> > > Using mmap has disadvantages: when you need to enlarge buffer text,
> > > and that fails (because there are no more free pages/addresses after
> > > the already allocated region), we need to copy buffer text to the new
> > > allocation.

64-bit address spaces are *huge*. What about just making every buffer 
allocation 2GB long or so, marked PROT_NONE? You don't actually have to 
commit all that memory --- all you've done is set aside that address 
space. But because you've set aside so much address space, you'll very 
likely be able to expand the actual allocation region (a subset of the 
reserved region) as much as you want.

> > All allocators suffer from this problem.  I haven't seen any evidence
> > that the mmap-based allocation code is significantly more prone to it.
>
> I have seen that.  The native glibc malloc, the on GNU/Linux systems
> were using until we got screwed by the recent glibc, didn't have this
> problem, while mmap-based allocator did.  Don't ask me how glibc does
> it, I don't know; but the fact is there.  This was discovered when the
> Windows mmap emulation in w32heap.c was developed and tested.
>
> > Also, the glibc allocators used mmap internally when allocating
> > large-ish chunks (e.g. for buffer text), so if that was a problem, we
> > would have noticed, I think.
>
> True; but they somehow work around the problem.
>
> > Another advantage of using mmap is that it can return the memory to the
> > OS once you kill your large buffer, whereas with gmalloc+ralloc this
> > basically never happens, AFAIK.
>
> Not entirely true: ralloc calls the system sbrk with a negative
> argument when it feels like it.

You can also madvise(MADV_DONTNEED, ...) regions *inside* the heap that 
contain only freed memory. This procedure also returns memory to the 
operating system.