Re: thread ids, task ids and subsystems

[Marcus Brinkmann]

On Fri, Apr 11, 2003 at 10:02:43PM +0200, Niels Möller wrote:
> Does this mean that if the server keeps one thread per cpu, it's best
> for clients to send the rpc requests to a thread running on a
> *different* cpu? You may save some working set migration, if you're
> lucky enough to target the right cpu, but on the other you need
> inter-cpu communication and both cpu:s are forced to do some
> scheduling.
> 
> If so, it seems better to me to keep the server thread on a single
> cpu: You'll use the caches better when most or all servers keeps their
> own working set on one cpu.

I am confused :)  But this is how I thought it would be:

The server keeps one thread per CPU.  All these threads do the same: They
wait for an incoming message (do a blocking receive), and when it arrives,
it dispatches it to a cache of worker threads (there is always at least one
worker thread, if not, the last worker thread first creates a new one before
processing the message).

There must be one cache of worker threads per CPU then, to make this model
work efficient.
 
> It's some time since I read about Solaris' ports concept, but IIRC the
> client thread would in some sense go on executing in the server, just
> in a different address space, and that was supposed to be pretty fast.

Well, this is a bit different.
 
> If I understood Marcus right, the idea was to target the rpc to a
> server thread on the *same* cpu as the client thread. Was that wrong?
> But we were also originally talking about the thread creation rpc, and
> I guess that's a kind of special case with extra scheduling
> requirements.

I don't really know if it is special.  I mean, if it's good for the thread
interface, it can't be bad for the other interfaces.

> Another subtlety with the one-thread per cpu model is that ports and
> reply ports must be treated differently.

A reply is always sent back to the sender.  I don't really see any issue
here.

BTW, yesterday I worked out a scheme which would allow multi-threaded
servers as well as single-threaded servers.  If the server thread id has the
lower N bits (which represent the processor number) all 0, this server would
then (by definition) be multi threaded and have one server thread per CPU
and the client must add the CPU number to the thread number part of the
thread id, whereas if the server thread id has some lower bits non-zero,
then it's single-threaded and the client should always sent to this one
thread, irregardless of CPU.

Unfortunately, this does not allow for processor sets.  If we were to allow
more flexibility, we would need a much more complicated scheme.

Thanks,
Marcus

-- 
`Rhubarb is no Egyptian god.' GNU      http://www.gnu.org    address@hidden
Marcus Brinkmann              The Hurd http://www.gnu.org/software/hurd/
address@hidden
http://www.marcus-brinkmann.de/