Re: wip-ports-refactor

[Chris Vine]

On Wed, 06 Apr 2016 22:46:28 +0200
Andy Wingo <address@hidden> wrote:
> So, right now Guile has a pretty poor concurrency story.  We just have
> pthreads, which is great in many ways, but nobody feels like
> recommending this to users.  The reason is that when pthreads were
> originally added to Guile, they were done in such a way that we could
> assume that data races would just be OK.  It's amazing to reflect upon
> this, but that's how it is.  Many internal parts of Guile are
> vulnerable to corruption when run under multiple kernel threads in
> parallel. Consider what happens when you try to load a module from
> two threads at the same time.  What happens?  What should happen?
> Should it be possible to load two modules in parallel?  The system
> hasn't really been designed as a whole.  Guile has no memory model,
> as such.  We have patches over various issues, ad-hoc locks, but it's
> not in a state where we can recommend that users seriously use
> threads.

I am not going to comment on the rest of your post, because you know
far more about it than I could hope to, but on the question of guile's
thread implementation, it seems to me to be basically sound if you
avoid obvious global state.  I have had test code running for hours,
indeed days, without any appearance of data races or other incorrect
behaviour on account of guile's thread implementation.  Global state is
an issue.  Module loading (which you mention) is an obvious one, but
other things like setting load paths don't look to be thread safe
either.

It would be disappointing to give up on the current thread
implementation.  Better I think in the interim is to document better
what is not thread-safe.  Some attempts at thread safety are in my view
a waste of time anyway, including trying to produce individual ports
which can safely be accessed in multiple threads.  Multi-threading with
ports requires the prevention of interleaving, not just the prevention
of data races, and that is I think best done by locking at the user
level rather than the library level.

Co-operative multi-tasking using asynchronous frameworks such as 8sync
or another one in which I have an interest, and pre-emptive
multi-tasking using a scheduler and "green" threads, are all very well
but they do not enable use of more than one processor, and more
importantly (because I recognise that guile may not necessarily be
intended for use cases needing multiple processors for performance
reasons), they can be more difficult to use.  In particular, anything
which makes a blocking system call will wedge the whole program.

Chris