Re: reentrant port table stuff

[Rob Browning]

Marius Vollmer <address@hidden> writes:

> The first major thing on the agenda is (in my view): do we want to
> care about full POSIX thread support, or are we satisfied with mere
> cooperative threads?
>
>>From an idealistic standpoint, I would say: within one address space,
> cooperative threading is best since it leads to a vastly simpler
> (low-level) programming model.  The concept of disallowing context
> switches is meaningful and a very cheap method to implement critical
> sections.  For us, this would mean that C code would not be
> interrupted without it taking actions to allow it (i.e. invoking
> SCM_TICK), and Scheme code could easily and cheaply prevent
> interruptions by setting some global flag.

In general I tend to agree that POSIX threading issues are *HARD*, and
that for most/many purposes involving guile, they're probably not the
most maintainable solution.  That said...

One concern I have with only supporting cooperative threads is whether
or not we can get the code "good enough" with respect to scheduling
fairness.  I've had enough experience with MacOS, MachTen, etc. to be
vary wary of cooperative coprocessing solutions for many jobs.  You
could argue that Guile's not designed for those jobs, but I'd really
like to be able to use it as widely as possible.

While implementing full blown guile threads on posix threads is a
possibility, as is only allowing cooperative threads, we might also
consider a middle ground solution.  I wonder if guile could be
implemented so that you could launch a guile interpreter
per-kernel-thread, and though each of these would not normally share
any memory, they could communicate through some well-defined "safe"
primitives (I'm somewhat thinking of Mach ports here).  NOTE: I
haven't considered this option for long enough to even know if it
passes the smell test, it just popped into my head :> Anyway, each of
these interpreters itself would still support cooperative threads
internally.

As you mentioned, one way to allow users to avoid the required
overhead involved in synchronizing preemtive threads would be to have
two versions of libguile.  That would let people pick whichever was
appropriate.  However I wonder whether or not we'd be able to handle
this cleanly in the codebase, or if we'd end up with painfully
divided maintenance issues.

As long as people can use a guile interpreter (or maybe interpreters)
from within an application that also has multiple kernel threads
running, possibly doing other things, I suspect that most of the
likely cases will be covered.

FWIW, Bill and I used threads in RScheme extensively, and its threads
were scheduled cooperatively with potential preemptions at every
function call and backward branch point, and as I recall, C code was
always atomic.  What was more important to us was for the IO to be
thread, but not process blocking.  i.e. other threads, kernel or
otherwise, needed to still be able to run when one thread was waiting
on socket or file IO.  Initially RScheme didn't support this, but we
helped add it.

> Ok, what do we make from this?  First I think we should remove old
> cruft, like the code covered by GUILE_OLD_ASYNC_CLICK, and
> streamline the async implementation.  Then, SCM_ALLOW_INTS and
> SCM_DEFER_INTS are no longer appropriately named.  They mark
> critical sections, and disabling interrupts does not save you from
> parallel access to global data structures.  We should rename them to
> something else or start using SCM_CRITICAL_SECTION_START /
> SCM_CRITICAL_SECTION_END instead.

This seems a reasonable way to start.  There's always the question of
whether or not we might want finer grained control later on (i.e. not
one global lock), but that can wait.

Also note that we learned the hard way that POSIX does not guarantee
fairness in it's mutexes/semaphores.  Starvation is quite possible.
If you want FIFO behavior, you have to build it on top if you want to
be portable.  At least that's what we found a while back.

> so that the memory is always known to the GC and can be freed when the
> above code sequence is aborted due to non-local exit.

So guile can exit a chunk of C code, "non-locally"?  Is there any good
documentation about path of control issues in guile?  i.e. I somewhat
presumed that C functions were "atomic", or are you saying that they
are atomic now, but wouldn't be with POSIX threads?

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD