Re: EROS/Coyotos fault handers vs l4 pager hierarchy

[Marcus Brinkmann]

At Fri, 21 Oct 2005 11:17:45 +0200,
Matthieu Lemerre <address@hidden> wrote:
> > We have established in previous emails that there tends to be a
> > hierarchy of pagers in L4. The root of this hierarchy is privileged
> > because it is the ultimate source of all pages.
>
> I don't think there is such a hierarchy in L4.

[...]

> What is hierarchic, though, is the mapping database of course.

Let's first establish for a fact that there is such a hierarchy.  In
our design so far, we have said that all memory is mapped from wortel
to physmem (irrelevant detail here), and then physmem maps it to the
individual processes.

The hierarchy here is: physmem at the top, and then every process as a
child node hanging down from it.  This is an extremely flat hierarchy,
but it is a hierarchy.  Moreover, the root (physmem) is privileged,
because it is the ultimate source of all pages, so it fits Jonathan's
description.

The mapping database is hierarchic, but it is not a concept of L4.  In
fact, the mapping database is internal to the kernel, it's not exposed
to the user at all.  It can only be modified implicitely by sending
map items over IPC.  However, please note: The L4 people consider the
in-kernel mapping database to be a _cache_ of the real mapping data,
which is distributed over the various pagers in the system (in our
case: physmem plus each individual pager in each task).  Thus, the
_real_ mapping database is tightly interconnected with the pagers in
the system.  Thus, the pagers in the system _tend_ to reflect the
hierarchical mapping database.

> > There is no comparable hierarchy in EROS. A fault handler's job is
> >to repair a page fault or not. Once a COPY is completed, there is no
> >further dependency on the sender's fault handler. In EROS, the
> >hierarchy is a matter of storage allocation, not mapping recovery.
> 
> I really think that there is no dependency on the sender's fault
> handler too on L4.  You can have a system which never invokes your
> pager (like the system I described in my last mails).

I think there is just a misunderstanding here.  The "sender of a page"
in our Hurd-on-L4 design for example is always physmem.  And any local
page fault (which can happen because physmem is allowed to revoke
mappings for internal reorganization) will be forwarded to physmem by
the local pager.  Thus, there is indeed a direct dependency.

There is also a second layer of abstraction, which is related to the
source of the actual data on a page, for example the filesystem.  But
this is independent of the paging mechanism.

> With the EROS approach, it seems to me that you can't customize the
> way swapping is done.

I am sure Jonathan will comment on this, but I can give a first hint.
There is the concept of "working sets", ie closely related pages.  I
don't know the details, but you can in some way influence the policy.
But not in a very fine-grained manner, this is true.  It's a problem
in EROS.  How big of a problem?  We have yet to find out.  This is a
discussion we need to have.

> What we originally planned to do was that
> applications could customize the way unused pages are sent to the
> swap, so that applications whose statistical page fault model doesn't
> fit well with the LRU algorithm could take advantage of it.  Would
> that still be possible with EROS?

As far as I know, no, and Jonathan is aware of the problem.  However,
he has argued in the past that we need a separation of paging
_policy_, which we want to be defined by the user, and page fault
_mechanisms_, which should be secured by the trusted computing base.

L4 does "err" on the side of policy, ie, it allows arbitrary policies,
but doesn't give you a secure mechanism.  EROS "errs" on the side of
mechanism, ie it has a secure mechanism, but does not offer arbitrary
policies.

Why does this matter?  For example, in the Hurd-on-L4 design, physmem
can be subject to DoS attacks.  In EROS, this is impossible, to my
understanding, as page faults can ever only be generated by the
kernel.  This is my preliminary understanding at least.
 
Thanks,
Marcus