--- Begin Message ---
|
Subject: |
Cuirass: The SQLite built in busy handler might block the Fibers scheduler |
|
Date: |
Sat, 21 Jul 2018 11:57:57 +0200 |
|
User-agent: |
mu4e 1.0; emacs 26.1 |
Hi,
I'm trying to understand why Berlin web API times out[1].
I think the SQlite built in busy handler may block the Fibers scheduler.
We use "PRAGMA busy_timeout = 30000;", which is an alternative to
calling sqlite3_busy_timeout(), whose description[2] is:
This routine sets a busy handler that sleeps for a specified amount
of time when a table is locked. The handler will sleep multiple
times until at least "ms" milliseconds of sleeping have
accumulated. After at least "ms" milliseconds of sleeping, the
handler returns 0 which causes sqlite3_step() to return SQLITE_BUSY.
To me this sounds like non-cooperative and non-resumable code.
A solution would be to set a custom handler through
sqlite3_busy_handler[3] that would be Fibers compatible, i.e. it would
let the scheduler schedule other fibers instead of just sleeping, using
Fibers 'sleep' procedure[4].
WDYT?
Clément
[1]: https://bugs.gnu.org/32233.
[2]: https://www.sqlite.org/c3ref/busy_timeout.html
[3]: https://www.sqlite.org/c3ref/busy_handler.html
[4]: https://github.com/wingo/fibers/wiki/Manual#user-content-index-sleep
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#32234: [PATCH 2/2] database: Serialize all database accesses in a thread. |
|
Date: |
Mon, 27 Aug 2018 15:47:27 +0200 |
|
User-agent: |
mu4e 1.0; emacs 26.1 |
Ludovic Courtès <address@hidden> writes:
> Hi Clément,
>
> Clément Lassieur <address@hidden> skribis:
>
>> Ludovic Courtès <address@hidden> writes:
>>
>>> Excellent, thanks for working on this! This looks great to me, and I
>>> think the pros outweigh the cons. Did you check on a big database how
>>> well it performs?
>>
>> Yes, I didn't see any difference. When I use Berlin's database, it
>> works well but crashes quickly for another reason (lack of disk space I
>> think, and /tmp being tmpfs).
>
> Sounds good (not that it crashes, but that you didn’t see any
> difference ;-)).
>
>>> I think I find it nicer to keep the ‘db’ parameter everywhere (except
>>> that it’s now a channel instead of an actual database) rather than using
>>> this global variable.
>>>
>>> WDYT?
>>
>> That 'db' parameter made sense before, because there were different
>> database connections: one per fiber. But now that there is only one
>> global channel accessible from everywhere, I can't find any use for a
>> 'db-channel' parameter.
>>
>> Also, using two differents channels for the same database would be a
>> bug, it would break the serialization mechanism.
>>
>> And I don't think using several databases (with one channel per
>> database) would make sense either.
>
> These are all good points, indeed. I’m mildly reluctant to the global
> parameter, but if you prefer it that way, I don’t mind; the end result
> matters more than this tiny issue anyway!
>
> So: LGTM.
>
> Thank you!
>
> Ludo’.
Ok, pushed!
--- End Message ---