Research using L4Hurd

[wtoomey]

Hi all,
        After surveying some of the recent OS research areas and available
vehicles (L4, DROPS, Mungi, L4Linux, Sawmill, L4Hurd), I think that L4Hurd
looks like a great candidate for academic research.

I mailed off to a masters student some details about my OS research idea,
which I've included below. I'd appreciate any feedback, comments, criticisms
on them.

What I'm not sure with L4Hurd is the current status, apart from being in
a discussion phase? Who's actively involved in the port, what are you guys
doing, is there a roadmap, will there be much change to the basic Hurd
architecture, what needs doing and how can I help?

Many thanks in advance for your replies.

        Warren


Research Idea
-------------
I'm interested in the SMP scaling problem, i.e to make OS scale nearly
linearly on SMP systems, but also to make them `buildable' i.e it is
humanly possible to build and understand one. Currently both Linux and
FreeBSD are getting stuck in the mire of removing the `Giant Kernel Lock'
from uniprocessor days and replacing it with fine-grained locking. See
http://www.bitmover.com/talks/cliq/slide01.html for such a discussion.

There is also the issue of making OS perform well on systems with
large caches, and the combination of caches and SMP makes it even harder.

I think the future lies with next-generation microkernels, see
http://www.l4ka.org/publications/files/towards-ukernels.pdf for a quick
review. Why:

        - the abstract OS services can be written using the standard
          process model, not weird quirky kernel code.
        - small footprint u-kernels fit into cache better and
          can deal with context switches faster.
        - syscalls, I/O etc get handled by things which can be blocked,
          which makes it possible to write abstract OS services using
          the standard process model.
        - we get away from the monolithic kernel design model, which
          requires lots of data structure locking, to a more
          component-based model with less locking but more IPC.
        - IPC is now inexpensive with u-kernels like L4.
        - probably lots of other reasons.

u-kernels should help to remove the SMP design complexity by removing
much of the locking, replacing it with well-defined IPC mechanisms.

What needs to be done?
----------------------
It seems possible that a full-blown OS (i.e with the large set of syscalls like
Linux or FreeBSD) could be built as a set of modular components on top of a
u-kernel like L4Ka. It seems possible to use IPC as a locking mechanism to
help scale the OS in an SMP environment. However:

        - no full-blown OS has been written as a set of modular components
          on top of a recent u-kernel. L4Linux was a port of Linux on top
          of L4, but the Linux was a single-threaded monolithic monster
          process which offered all the Linux syscalls, and so would perform
          miserably in an SMP environment.
        - it is still unclear as to whether a u-kernel will help the SMP
          scaling issue. It entirely possible that a u-kernel approach will
          just move the locking-scaling problem down into the IPC design
          and implementation.

So, a long-term project would be to:

        - write or port a full-blown OS to run on top of a recent u-kernel,
          but in a modular componentised fashion which would thus be
          multi-threaded and thus would be SMP-ready.

          There is an effort to port the Hurd by removing its dependency on
          the Mach mega-microkernel, and to run it on top of L4. The Hurd
          is designed as a set of modular components, but there is a big
          dependency on Mach. Lots of discussion at present, no code yet.

          Alternatives would be Minix, Linux, FreeBSD etc. The latter two
          have more services, but you would have to undo the monolithic
          structure, which would be very difficult. Linux would probably
          be easier, as it has awell-defined division between
          hardware-dependent and hardware-independent layers. Minix
          would be the easiest to port, as it is already modular, but
          is more of a toy OS.

        - with a full-blown OS on top of a recent u-kernel, begin the process
          of measuring SMP performance and SMP scalability. Find if there
          are design/architecture problems, or design/implementation problems
          and if the idea is still valid, tune and/or redesign to make it work.

This is a large project with truly academic results not becoming available
until the second phase, although there would be some (mostly non-SMP)
academic results coming out of the first phase.