Re: A comment about changing kernels

[Jonathan S. Shapiro]

On Tue, 2005-11-01 at 01:22 +0100, Bernhard Kauer wrote:
> > we may safely conclude that these CALL/RETURN patterns
> > describe the overwhelming majority of IPCs -- all of the other patterns
> > taken together accounted in EROS for less than 1% of all dynamic IPCs.
> 
> Is the CALL/RETURN pattern used for capability delegation (giving an cap
> to another party) or only for usage of these capabilities?

For delegation, we tend to see a CALL/RETURN pair in which one of the
arguments to the CALL is the capability being delegated.

However, these delegations are dynamically very rare in comparison to
*uses* of capabilities. As an imperfect intuition: delegation tends to
occur during program setup, but rarely during program steady-state
execution. A delegation:use ratio of 1:100 or even 1:1000 would not be
an unreasonable expectation.

> > As we considered the session/sessionless issue in Coyotos, I became
> > convinced that using the protected payload as a session ID would not
> > scale well, and also presents certain challenges for bootstrapping and
> > isolation.
> 
> Interesting. What are the scalability and isolation problems with badges?

Remember that my opinion was based on the old, extensible badge idea. At
the time, there were several bits of protocol being considered whose
efficiency relied on being able to extend the badge, but with a 32-bit
(or even 64-bit) badge, there really wasn't any reason to feel confident
that you had "enough" badge bits to work with, and I could never find a
clear story for what to do when the badge bits ran out.

Since you now say that badges are no longer extensible, I would guess
that other protocols have been developed that avoid this issue and/or
solve it in other ways.

> > It involves a cache-aligned four-word move from one process to the other,
> > using a scaled index addressing mode using the respective process structure
> > pointers as base pointers.
> 
> This is a cache miss and perhaps a TLB miss since you have to touch the 
> process
> structure (I assume a thread/task split like in L4).

There is no task/thread split in EROS or Coyotos, so there is no
additional TLB miss. We have only threads, and no tasks. Address spaces
are first-class. Each thread holds a capability to its address space;
two threads can hold capabilities to the same address space.

> But don't you allow the revocation of this capability later?

Not in general. In EROS, this is by far the unusual case (much less than
1/100,000 of all delegations).

> > I am *guessing* that this is significantly lower than the cost that you
> > anticipate for the MAP operation.
> >
> > Second question: do you agree that this difference in designs would
> > largely explain our differing views about the costs of a sessionless
> > protocol?
> 
> Yes, but up to now I am not convinced that you can build a revocable copy
> with just a single cache miss.

That is probably correct, but that is not the goal. In fact, quite the
opposite is desired. We consider revocable copy a necessary evil that
should be avoided whenever it is possible to do so -- there has been a
long discussion on this list over the last few weeks about why we have
this view.

> > If the kernel does not support the sessionless protocol, then the server
> > must execute the lookups necessary to map from session identifiers to the
> > appropriate endpoint capabilities. The cache and TLB misses associated
> > with this activity must be accounted as part of the IPC costs in order
> > to make an accurate comparison.
> 
> The session-id -> return-endpoint lookup in user-level could be the identity
> function, since the server is free to manage the badges. The following
> capability-space lookup could be more expensive, but this is optimizable
> with capability registers...

There exists in the world an infinite set of hypothetically feasible
optimizations. This is not interesting. The interesting questions are:

  1. What patterns are going to get used in realistic practice, and
  2. What is their cost.

I agree that the badges can be managed successfully in some servers, but
the optimization you suggest does not look like a general design pattern
(because badges have *other* uses that are important).

I personally tend to believe that hand-customizing servers is something
that should be a last resort, because these customizations become an
ongoing engineering burden.

But even if the lookup cost is zero, the C-space traversal cost is
likely to be significant. If the sessions have long lives, locality and
caching of the C-space entries would seem to be tricky.

> > And in fact, if you look at the EROS window system, you will discover
> > that it is sessionless. So is our current ethernet driver (though this
> > one needs to change).
> 
> Both components are session based in DROPS.

Strictly speaking, there is a small amount of session state in EWS as
well. We consider this a bug that we would like to repair. I was typing
too quickly. What I should have said is that EWS does not store any
per-session capabilities.

shap