[U]VM subsystem for L4 X.2 (and later)

[Farid Hajji]

Hi,

we need a Mach-VM like system for L4 which can be used by
Hurd/L4 and *BSD/L4 alike. The goal is to implement a
"vmserver" L4 task that embodies the VM subsystem from
one of BSD VM, Mach VM, FreeBSD VM or [Open- and Net]BSD
UVM and that provides its services to clients through
L4 IPCs.

'vmserver' would not be a complete Mach 'server' like
what Okuji suggested, but only a subset thereof. It is
needed, because the Hurd (and BSD) heavily rely upon a
vm_*() interface and L4's memory management philosophy
of recursive pagers is sufficiently different from VM
that we need to bridge the gap somehow.

There are some VM systems we can choose from:
  1. [4.4]BSD VM
  2. Mach VM
  3. FreeBSD VM
  4. UVM (NetBSD and OpenBSD)
  5. Linux VM

1. is historical and was subsequently modified in 3. and 4.  so that
we don't need to bother too much about it.

2. would be obvious from the standpoint of the Hurd, but this is not
necessarily true. 2. suffers from the same disease than 1. and (to a
limited extent 3.) termed the "vm_object shadow chain problem". We
_could_ try to encap oskit-mach's or gnumach's VM subsystem into
vmserver, but I'd suggest that we keep only the external pager
interface while using 4. for the internals.

3. worked around some of the limitations present in 1. and 2., but
still has to use very complex shadow chain collapsing algorithms which
means higher code complexity to deal with.

4. is IMO the best alternative right now, because it doesn't have the
'shadow object chain' problem anymore and because of its very clean
separation of MI and MD code (which is very important for the L4 port
here).

5. doesn't keep enough state _outside_ the page tables and PTEs. This
makes it difficult to adapt it to L4 X.2. Sure, we can look at the way
how L4Linux is implemented, but I doubt that we can port this to
"vmserver" in a sensible way.

So at the moment, I'm preferring UVM over the other alternatives,
though I didn't made up my mind yet.

The UVM system consists of 2 parts:
  a. The MI UVM code
  b. The MD pmap module

a. manages the bookkeeping of the VM data structures in a machine
independent way. It decides which pages should be made available,
which should be COWd etc...

b. programs the MMU in a machine dependend way (remember: NetBSD runs
on a _lot_ of different architectures!). The UVM <-> pmap interface is
the same across all supported architectures, so all we need to do is
to write yet another pmap module for L4.  This pmap/l4 module would
use L4 IPCs to implement its services just like pmap/{i386,m68k,...}
uses CPU instructions/registers.

So, the issue of an L4 pmap seems resolvable, both for current L4
(say: hazelnut) and for the upcoming Version 4 spec. It is just a
matter of using L4's memory mapping primitives inside a new pmap.c to
implement the functions required in pmap(9).

The next issue in dealing with UVM/L4 is how to access swap device
without any disk driver. UVM uses the vnode layer of BSD to access the
drum/swapfile, so this is a kind of recursive strange loop back into
the BSD kernel. We'll have to think more about it for everything else
than the BSD/L4 port, because we won't have that vnode interface
(yet). So how do we access swap then? A first quick hack would be to
ignore the problem completely by short-circuiting the vnode operations
and use reserved memory for backing store [sort of: memory-based
swap-file ;-)]. In a later stage, when we have device drivers for L4
(or L4Env), we can still reactivate that vnode layer of 'vmserver'.

Another issue to deal with, is locking of data structures in UVM
itself. UVM is multi-threaded and thread-safe (by using fine grained
locks), but it could be difficult to adapt this to L4 environment. I
didn't thought about it yet, so feedback is highly welcome.

If someone is intersted in helping with '[u]vmserver', please coordinate
with me. I didn't start any coding yet (still studying UVM code), but
I plan to start as soon as I have clear enough understanding of how
UVM, pmap and the rest of the system fits together.

Regards,

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
One OS To Rule Them All And In The Darkness Bind Them... --Bill Gates.