Re: Comparing "copy" and "map/unmap"

[Jonathan S. Shapiro]

On Tue, 2005-10-11 at 01:15 +0200, Matthieu Lemerre wrote:
> "Jonathan S. Shapiro" <address@hidden> writes:

> > But I think that the real difference lies in the fact that our system
> > tends to favor designs where storage used by a server is allocated from
> > a homogeneous source. This greatly simplifies matters.
> 
> What do you mean by homogeneous source?

I mean that in most of our objects, all of the storage comes from a
single storage allocator. As long as there is only one source of
storage, there is no complexity managing it. The complexity only begins
when you must manage storage from more than one source.

> > I think I have said previously that storage management is all handled at
> > user level. I meant that quite literally. The kernel does not do
> > allocation.
> 
> If I understand you correctly, then for each allocated object on a
> server, a client has to allocate a whole page frame and supply it to
> the server?

No, at two levels:

First, a problem of small details: the majority of servers in our system
are "single client". The entire server has an exclusive arrangement with
its client, and it uses a storage allocator provided at server creation
time. I will come back to this in my next note.

Second, the majority of servers allocate from a common source of
storage.

But yes: if objects are not supposed to share state, then the minimum
unit of data allocation is a page.

> If it is so, isn't it a waste of memory for capabilities on objects
> that don't need that much data?  If we take, for instance, a time
> server, whose objects have the current time recorded upon creation
> (this is a theoretical example).  Should then the client allocates one
> whole page for the three objects it wants to create?

Yes. It is a waste of storage. All 14 of the objects in your entire
system that are this small will waste nearly a page each. Real objects
are big -- but let me come back to this too in just a moment below.

The time server is not a good example. In the case of the time server,
the right thing is for the server to allocate a *single* page from
*server* storage, put all three times in it, and simply hand out
read-only capabilities to that page. All clients access the same page.

Okay: back to your concern about small objects.

The issue you are concerned about is only an issue in two cases:

  (1) Storage used by the server is not homogeneous, or
  (2) Clients need shared memory access to their objects,
      and the objects are tiny.

Both issues are individually rare. In fact, we have no examples of
either.

We do have servers that implement a large number of small objects. It
turns out that *all* of these servers use homogeneous storage, so they
do not allocate storage on a per-object basis. What happens in the
homogeneous case goes like this:

1. Some client holds a capability to the server that allows it to
request new objects using an IPC.

2. The server allocates this object out of server memory. This memory
comes from a space bank that was supplied at server creation time. The
server takes a pointer to this object.

3. The server now takes a second, distinguished capability to itself,
and places this capability into a newly allocated wrapper object. This
second capability is the one you call to get per-object behavior.

4. The server *also* stores into the wrapper a pointer to the
in-server-memory location of the object Whenever this wrapper is
invoked, the stored pointer will be provided **in addition to** any
arguments supplied by the client. Note that the client of a wrapper
cannot examine this pointer. It is protected payload.

For servers implementing a smaller number of objects that do not need to
be individually revocable we could just use the protected payload in the
endpoint capability. The reason to use the wrapper is so that objects
can be revoked individually.

So in practice, the space for an object is the space for a wrapper plus
the in-memory space for the object itself inside the server address
space.

> (Sorry if my question seems dumbly simple, but I didn't manage to find
> something that explicates in detail how this works).

The question is not dumb at all. The EROS documentation is better than
any other research system I know about, but it isn't very good.

shap