bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler

[Clément Lassieur]

Ludovic Courtès <address@hidden> writes:

>> I think the SQlite built in busy handler may block the Fibers scheduler.
>> We use "PRAGMA busy_timeout = 30000;", which is an alternative to
>> calling sqlite3_busy_timeout(), whose description[2] is:
>>
>>     This routine sets a busy handler that sleeps for a specified amount
>>     of time when a table is locked. The handler will sleep multiple
>>     times until at least "ms" milliseconds of sleeping have
>>     accumulated. After at least "ms" milliseconds of sleeping, the
>>     handler returns 0 which causes sqlite3_step() to return SQLITE_BUSY.
>>
>> To me this sounds like non-cooperative and non-resumable code.
>
> Indeed!
>
>> A solution would be to set a custom handler through
>> sqlite3_busy_handler[3] that would be Fibers compatible, i.e. it would
>> let the scheduler schedule other fibers instead of just sleeping, using
>> Fibers 'sleep' procedure[4].
>
> AIUI the handler is called from C, and thus it’s a non-resumable
> continuation, so this wouldn’t work.

Oh, I see.

> Perhaps instead we need to set the timeout to a small value and handle
> SQLITE_BUSY at the call site in our code.  We could define a macro that
> automatically retries upon SQLITE_BUSY.

That would limit the issue to the first timeout span: for that short
time the scheduler would be blocked.  I think a timeout of 0 would be
better.

Another solution would be to serialize all the database accesses as we
do already with the url handler, and stop using the SQLITE
multithreading features.  It would probably make the code simpler
because we would use the same paradigm everywhere, and we would avoid
looping until SQLITE isn't busy at each request.

WDYT?

Clément