Re: Comments on the hurd-on-l4 document

[Niels Möller]

Marcus Brinkmann <address@hidden> writes:

> At 08 Jun 2005 20:24:33 +0200,
> Niels Möller wrote:
> > I think the notification server at least is fairly simple (although it
> > will be on the communication path, adding overhead). What about this
> > minimalistic notification protocol, somewhat inspired by amigaos
> > "signals" (which are quite different from unix signals):
> 
> What you describe is pretty much what I came up with, too, with some
> extensions:
> 
> * I was considering to let the caller hand out memory to the
>   notification server where the notifications can be stored.  IE, the
>   user must provide the memory resources via containers.  If the user
>   revokes the memory, it loses the notifications, tough.

I was trying to think minimalistically. Since the notification server
will be a single central almost mandatory service, it shouldn't
implement any features that could just as well be implemented by it's
users. (Just like features are kicked out of L4 if they can be
implemented reasonably in userspace).

The signaling abstraction seems to match that. Requirements on "real"
notifications will be very different for different applications and
protocols, e.g.

* one shot notifications capabilities vs n-time vs notification
  capabilities which are valid until explicitly revoked.

* notifications when there's no need to distinguish between a single
  notification and multiple notifications within a short time, vs
  notifications where delivery of every individual notification is
  important.

* small vs large amounts of extra data

* need for notification queuing, with various policies such as a
  maximum queue size, message priorities, whatever.

They are all straight forward to implement using the signal primitive
I sketched. Let's consider a few of them briefly.

* One-shot: After receiving the notification, revoke the capability,
  and send a message to the notification server to clear the bit.
  Preferably before before asking for the next notification. (Having
  the notification server associate a counter or flag which each
  capability is sufficiently simple that I think it might be a god
  idea, but it's not necessary; the receiver can do that just fine).

* When delivery of every single notification is important: Let the
  server that sends the notification maintain a counter incremented
  for each call to the raise rpc. When signaled, the receiver asks the
  server for the current value of the counter, and then acts on the
  difference since the last time (which might be zero, depending on
  timing. It's important that the server increments the counter
  *before* raising the signal ).

* Extra data: Store the data in the server, let the client read it
  after receiving the signal. The server will typically already have
  the data. And I don't think that sending the data via the
  notification server is a worthwhile optimization, (and it can only
  ever be an optimization if there's very little data and no magic
  types, otherwise the overhead of an extra rpc on the data path
  becomes significant).

* Queues and stuff: Like other cases of extra data, the queue should
  live in the server. If desirable, it can use memory provided by the
  client. Sometimes, it might even be possible to put the queue into
  memory shared between client and server, which definitely beats
  having a queue in the notification server.

One can think of the signalling primitive as a way to link two
condition variables in different address spaces together. And
condition variables are powerful enough to implement all sorts of
fancy communication mechanisms.

One more thing: If we want to minimize memory usage, one could give
clients a small default signal and capability allocation such that the
data fits in a small fraction of a page. Tasks that needs more signals
than that has to provide one or more pages to the notification server
for the data; a single page should be sufficient for most tasks. (I
would even prefer not to have the notification server have to work
much with containers; perhaps it's possible to use a raw map/grant
rather than a container here. Don't know if that's feasible though;
the crucial part is that the notification server must be able to
destroy all corresponding capabilities if there's a page fault. Note
that much more storage will be needed for the capability book-keeping
than for the actual signalling bitmaps).

One has to think carefully before considering paging out the
notification server, so I think it is quite important to keep it
minimal in both code and data.

Regards,
/Niels