[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: When should ralloc.c be used?

From: Daniel Colascione
Subject: Re: When should ralloc.c be used?
Date: Fri, 28 Oct 2016 08:41:48 -0700
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0

On 10/28/2016 05:25 AM, Eli Zaretskii wrote:
Cc: address@hidden, address@hidden, address@hidden
From: Daniel Colascione <address@hidden>
Date: Fri, 28 Oct 2016 02:52:19 -0700

If reserving a range of addresses doesn't necessarily mean they will
be later available for committing, then what is the purpose of
reserving them in the first place?  What good does it do?

Reserving address space is useful for making sure you have a contiguous
range of virtual addresses that you can use later.

But if committing more pages from the reserved range is not guaranteed
to succeed, I cannot rely on getting that contiguous range of
addresses, can I?

You already _have_ the range of addresses. You just can't do anything with them yet.

Here's another use case: magic ring buffers. (Where you put two consecutive views of the same file in memory next to each other so that operations on the ring buffer don't need to be split even in cases where they'd wrap the end of the ring.)

Say on our 1GB RAM, 1GB swap system we want to memory-map a 5GB ring buffer log file. We can do it safely and atomically like this:

1) Reserve 10GB of address space with an anonymous PROT_NONE mapping; the mapping is at $ADDR 2) Memory-map our log file at $ADDR with PROT_READ|PROT_WRITE; (the mapping is file-backed, not anonymous, so it doesn't count against system commit charge)
3) Memory-map the log file _again_ at $ADDR+5GB

Now we have a nice mirrored view of our ring buffer, and thanks to the PROT_NONE mapping we set up in step one, no other thread was able to sneak in the middle and allocate something in the [$ADDR+5GB,$ADDR+10GB) range and spoil our ability to set up the mirroring.

In this instance, setting aside address space without allocating backing storage for it turned out to be very useful.

We have in w32heap.c:mmap_realloc code that attempts to commit pages
that were previously reserved.  That code does recover from a failure
to commit, but such a failure is deemed unusual and causes special
warnings under debugger.  I never saw these warnings happen, except
when we had bugs in that code.  You seem to say that this is based on
false premises, and there's nothing unusual about MEM_COMMIT to fail
for the range of pages previously reserved with MEM_RESERVE.

The MEM_COMMIT failure might be rare in practice --- systems have a lot
of memory these days --- but MEM_COMMIT failing for a memory region
previously reserved with MEM_RESERVE is perfectly legal.

I can only say that I never saw that happening.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]