Re: When should ralloc.c be used?

[Daniel Colascione]

On 10/28/2016 05:25 AM, Eli Zaretskii wrote:
> > Cc: address@hidden, address@hidden, address@hidden
> > From: Daniel Colascione <address@hidden>
> > Date: Fri, 28 Oct 2016 02:52:19 -0700
> >
> > > If reserving a range of addresses doesn't necessarily mean they will
> > > be later available for committing, then what is the purpose of
> > > reserving them in the first place?  What good does it do?
> >
> > Reserving address space is useful for making sure you have a contiguous
> > range of virtual addresses that you can use later.
>
> But if committing more pages from the reserved range is not guaranteed
> to succeed, I cannot rely on getting that contiguous range of
> addresses, can I?

You already _have_ the range of addresses. You just can't do anything 
with them yet.

Here's another use case: magic ring buffers. (Where you put two 
consecutive views of the same file in memory next to each other so that 
operations on the ring buffer don't need to be split even in cases where 
they'd wrap the end of the ring.)

Say on our 1GB RAM, 1GB swap system we want to memory-map a 5GB ring 
buffer log file. We can do it safely and atomically like this:

1) Reserve 10GB of address space with an anonymous PROT_NONE mapping; 
the mapping is at $ADDR
2) Memory-map our log file at $ADDR with PROT_READ|PROT_WRITE; (the 
mapping is file-backed, not anonymous, so it doesn't count against 
system commit charge)
3) Memory-map the log file _again_ at $ADDR+5GB

Now we have a nice mirrored view of our ring buffer, and thanks to the 
PROT_NONE mapping we set up in step one, no other thread was able to 
sneak in the middle and allocate something in the [$ADDR+5GB,$ADDR+10GB) 
range and spoil our ability to set up the mirroring.

In this instance, setting aside address space without allocating backing 
storage for it turned out to be very useful.

> > > We have in w32heap.c:mmap_realloc code that attempts to commit pages
> > > that were previously reserved.  That code does recover from a failure
> > > to commit, but such a failure is deemed unusual and causes special
> > > warnings under debugger.  I never saw these warnings happen, except
> > > when we had bugs in that code.  You seem to say that this is based on
> > > false premises, and there's nothing unusual about MEM_COMMIT to fail
> > > for the range of pages previously reserved with MEM_RESERVE.
> >
> > The MEM_COMMIT failure might be rare in practice --- systems have a lot
> > of memory these days --- but MEM_COMMIT failing for a memory region
> > previously reserved with MEM_RESERVE is perfectly legal.
>
> I can only say that I never saw that happening.
>
> Thanks.