Re: self-paging

[Bas Wijnen]

On Thu, Dec 01, 2005 at 07:30:28PM -0500, Jonathan S. Shapiro wrote:
> I apologize for my delay in responding on this thread -- I've been
> dealing with impending final exams and some Coyotos contract issues.

I missed you. ;-)  I hope everything went well with the exams and the
contracts.

> Concerning self-paging, I think we need to break the discussion down
> into several distinct issues:
> 
>   1. Hard vs. soft page frame commitments
>   2. Application control over pageout selection
>   3. Notifying the application about changes in contract
> 
> The first point is simple. I just wanted to point out that we are
> talking here about negotiated, changing contracts, so we are discussing
> only "soft" contracts.

Right.

> The point: you don't need to notify the application of anything in order
> to let the application define residency requirements.

Well, it's all fine when the quota decreases.  A page will be swapped out, and
the process will get a page fault at some point.  In response to that it can
reoptimise its strategy (how much cache to have, for example).

However, this doesn't work when the quota increases.  Assume a process which
wants as much cache as possible, but only in physical memory.  When it knows
how much it can use, it will allocate that and use it.  As long as it isn't
notified, it doesn't start using more, even if that would be more efficient.
Of course it could poll for its quota every now and then, but that doesn't
seem right.  It just adds work to the process while all the information to
notify it is available.

> The notification issue is a problem, because it is a universal
> communication channel among all processes. There are two different
> models to consider here:
> 
>   1. Soft reservations are adjusted dynamically by some agent in
>      response to pressure. This is the case where we need to
>      consider channels.

We always need to consider channels, except with hard page frame commitments.

>   2. Soft reservations are adjusted only in response to user action.
>      For example, when your window ceases to be in front, we can tell
>      you safely that your CPU contract and memory contract are reduced.

While this would not be an acceptable policy for me, I agree that the idea can
work.  I'm not sure if I'd like it though.  I think I prefer to have settings
for the quota which can be bound by the user (so dynamic values, but limited
to a range).

>       This does not create a channel issue,

It does.  The channel is just very low bandwidth.  But if a process runs for
days, and some program is so slow at certain actions that the user will go do
something else while it's working, then that is noticable by other processes
and it can be used as a communication channel.

>       and it seems to deal very well with the problems that people are
>       really trying to solve when they do this type of adaptive reservation.

The policy you gave (foreground == highest priority) is very single-tasking
oriented.  I like to do things in the background, and I wouldn't like it if
those things get less resources just because I'm typing in an other window.  I
think this is not just a nerd-thing.  On Windows, people learned not to
multi-task, because it may crash as a result ("make sure you close all other
programs while the installation is running").  But when I show people that on
GNU/Linux they can, they are very eager to do it (although not too confident
that it will not crash at first).

What your reply didn't comment on is the two big parts of my proposal:

- The application manages a list of its pages, thereby it can do its own
  paging (literally while the quota don't change, only preparing the order
  when they do).  What I didn't mention yet, is that I fear this may cause
  overhead (because the list must be managed with system calls to physmem).
  Perhaps this isn't a problem if the management is simply ignored until
  swapping starts.  What that happens, RPC performance is no longer a
  significant factor anyway.
- Change the quota very slowly, not immediately on every event, so the
  communication channel is narrowed.

What is your opinion about these?

I did not go into the problems of how to set the quota.  That's a separate
discussion.

Thanks,
Bas

-- 
I encourage people to send encrypted e-mail (see http://www.gnupg.org).
If you have problems reading my e-mail, use a better reader.
Please send the central message of e-mails as plain text
   in the message body, not as HTML and definitely not as MS Word.
Please do not use the MS Word format for attachments either.
For more information, see http://129.125.47.90/e-mail.html

signature.asc
Description: Digital signature