Re: A short note on microkernels

[Jonathan S. Shapiro]

This will sound strange coming from me, but it may turn out that HurdNG
should build it's own microkernel. I do not think this is a good idea,
but let me try to enumerate the alternatives and articulate the
advantages and disadvantages as I see them.

In abstract, there are three options:

  1. Adopt an existing microkernel.

  2. Adopt an existing microkernel, possibly with extensions
     or modifications.

     By this I mean "extensions or modifications that the originators
     cannot be persuaded to adopt"

  3. Build a new microkernel from scratch.

Option 1: Adopt an Existing Microkernel

Pros:

  1. Benefit from a great deal of work that is already done and tested.
  2. Benefit from wider use -- other users of the microkernels will
     find and fix bugs for you.
  3. No need to design a (micro)kernel from scratch, which is very hard.
  4. There is a supporting team that exists outside of Hurd resources.
  5. Saves about 2-3 years.

Cons:

  1. No microkernel will ever be a perfect match. A good microkernel
     is general purpose, and therefore will not be "perfect" for Hurd.
     However, this is deceptive, because it is very very hard to do
     better in practice.
  2. There is a supporting team you need to interact with, and the Hurd
     may not be their primary objective.

Option 2: Adopt an existing microkernel with extensions or modifications

Pros:

  1. Benefit from a great deal of work that is already done and tested.
  2. *Some* benefit from bug fixes in the base system, but this benefit
     will tend to decay quickly.
  3. Saves about 1-2 years.

Cons:
  1. All Cons from (1) apply here.
  2. No team supporting you, because you are a deviant community. This
     creates both social issues and cost issues even if the original
     maintainers want to work with you.
  2. Unlikely to work in practice -- see comments below.

Option 3: Build a New Microkernel

Pros:

  1. Could be a perfect match for your problem.

Cons:

  1. Costs about 3-4 years.
  2. Requires very specialized expertise.
  3. No external support team to turn to.
  4. No synergy with other users.


The tempting position is to adopt position (2). Let me explain why it
rarely works in practice.

Mature microkernel designs arrive at a sort of a "least energy" design
state. This is the state where (a) there is a cohesive design, (b)
almost everything "fits" together, and (c) the consequences of changing
one thing lead to changes in everything else. I have sometimes said
about EROS that "problems exist, we know where they are, and we have
reached the point where they cannot be eliminated -- only pushed around
in the system or made to manifest with different symptoms."

A consequence of this is that "enhancements" divide in a fairly black
and white way into:

 1. Non-invasive enhancements. Unless these are silly, the originating
    team will either accept them or explain why they are a bad idea.

 1. Major changes that the originators accept. Example: Coyotos FCRBs
    and the activations stuff, which are Neal and Marcus's fault. :-)

 2. Invasive enhancements that the originators reject. These have
    cascading consequences, and it becomes very important to understand
    that they really are a good idea. They are unlikely to get adopted
    by the originators, because they are rarely compatible and they
    involve a large (therefore risky) change to the design.

The third category basically means that you are designing your own
kernel while borrowing a lot of stuff from an existing kernel. If you
need to build a new kernel, this is a good way to go, but the cost of
doing things this way is about 10x what people naively estimate.



So what, in my opinion, should Hurd-NG do?

1. Check out the existing (or forseeable) alternatives very carefully.
Try to find an existing kernel you can live with. The answer will be
"no". Now try again. See if you can get the new things you want
accepted.

2. Look at your resources and expertise. Even if you *want* to build a
new (micro)kernel, how realistic is it to expect that you can do so
successfully?

3. Find a provider who wants you to succeed and will work with you. I'm
trying to work with Neal and Marcus, but there are still unresolved
issues in Coyotos relative to what Hurd needs. In some cases we agree
about the need but we do not have a suitable specification yet. In other
cases we don't quite know what we want yet. Most of these will get
resolved if Hurd uses Coyotos. Some will not.

4. Be very skeptical if somebody says "building a microkernel is easy".
Here is an observation: in 1990 there were about 15 research
microkernels around. Today there are only two (L4/L4.sec and
EROS/Coyotos). This is mostly because the others could not compete or
were not maintained.

If, after you look at the alternatives, you still think that building or
modifying a microkernel sounds good, try to assemble a team. If you
can't, stop and look for another way.

In my opinion, the L4-Hurd and mainstream Hurd communities simply do not
have the (human) resources to build both a microkernel and an OS before
most of this list has died of old age.

> There is a natural desire to start with a solid foundation and build
> on top of it.  However, a solid foundation also restricts the number
> of options that can be seen as reasonable.

I agree completely. This is because of the "least energy" effect.

However, it is also natural to say "I can do better." Too often, this
leads to an unsound foundation or a project that never gets completed.


shap