Re: February update on the Guile guix-daemon

[Christopher Baines]

Ludovic Courtès <ludo@gnu.org> writes:

> Christopher Baines <mail@cbaines.net> skribis:
>
>>  - Then there's the big areas to work on next:
>>
>>    - I think I'm going to need to use thread pools for SQLite operations
>>      in the daemon, as the build coordinator does.
>
> I think we should refrain from using POSIX threads directly and instead
> use Fibers to its full extent.  In this case, I’d use a resource pool as
> done in Cuirass (and in the Coordinator too, no? maybe that’s what you
> meant?).
>
>>    - There's the low level work of setting up the build environment, the
>>      work on the guile-daemon branch helps a lot with this, but as
>>      pointed out by Ludo, there might be some issues with fork and
>>      similar operations in a Guile program using threads.
>
> Right.  I think one of the things we discussed in Brussels is that,
> since SQLite operations might block for a while, the daemon will have to
> be multithreaded.  Which means no fork/clone, which in turn probably
> means delegating forking/cloning to a separate helper process.  (That’s
> also what bubblewrap does, IIUC.)

I've had a bit more of a think and have thought of some more things we
can try, but I'm still not sure if it can be made to work.

We can call (sleep 0) after every sqlite-step to allow other fibers to
run if they're able to [1] and if we find places where sqlite-step takes
too long, maybe we can use a short lived thread for this, but then make
sure these threads have stopped before doing anything where they would
cause problems (e.g. fork/clone).

1: https://github.com/wingo/fibers/wiki/Manual#31-blocking

signature.asc
Description: PGP signature