bug#31878: Module autoloading is not thread safe

[Mark H Weaver]

Hi Ludovic,

address@hidden (Ludovic Courtès) writes:

> Mark H Weaver <address@hidden> skribis:
>
>> Since Guile (unfortunately) allows cyclic module dependencies, we would
>> need a mechanism to avoid deadlocks in case modules A and B both import
>> each other, and two threads concurrently attempt to load those modules.
>>
>> The first idea that comes to mind is to also have a global structure
>> storing a partial order on the modules currently being loaded.  If,
>> while module A is being loaded, there's an attempt to auto-load module
>> B, then an entry (A < B) would added to the partial order.  The partial
>> order would not allow cycles to be introduced, reporting an error in
>> that case.  In case a cycle would be introduced when adding (A < B),
>> then the thread would simply be given access to the partially-loaded
>> module B, by adding B to its local list of modules-being-loaded.
>
> Would it enough to (1) use recursive mutexes, and (2) have
> ‘resolve-module’ lookup modules first in the global name space, and
> second in the local list of modules being loaded?

Item (2) above is something that I had already envisioned in my
proposal, although I neglected to mention it.

However, I don't see how recursive mutexes would help here, or how they
could obviate the need for the other mechanisms I described above.

Suppose module A and module B are mutually dependent on each other.  If
thread 1 is loading module A concurrently with thread 2 loading module
B, then thread 1 will be the only thread with access to module A (via
thread 1's local list) and will hold the lock on it, and similarly for
thread 2 and module B.

Now, when thread 1 tries to load module B (while it's in the process of
loading module A), it should normally be blocked until module B is
finished loading.  If those modules were _not_ mutually dependent on
each other, we should insist on thread 1 waiting for module B to finish
loading before gaining access to it.  Only if there is a cyclic
dependency should it be granted access to the partially-loaded module.

If we simply use recursive mutexes, I think deadlock would occur in this
case.  Thread 1 would try to grab the lock on module B, which is already
held by thread 2, and vice versa.  Since it's not self-held, I fail to
see the relevance of the recursive mutex.

What do you think?

      Mark