[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#42001: “SQLite database is busy”: contention on the store database

From: Ludovic Courtès
Subject: bug#42001: “SQLite database is busy”: contention on the store database
Date: Tue, 23 Jun 2020 09:42:02 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)


Mathieu Othacehe <othacehe@gnu.org> skribis:

>> utimensat(AT_FDCWD, 
>> "/gnu/store/whhc80cx1zpi9cj3bil3sp8wwzcf8p9p-image-root/gnu/store/n6l40ygclxfysnc2nxwrslnn5ab6wv3v-profile/lib/modules/5.4.47-gnu/kernel/net/netfilter",
>>  [{tv_sec=1592814617, tv_nsec=396083395} /* 
>> 2020-06-22T10:30:17.396083395+0200 */, {tv_sec=1, tv_nsec=0} /* 
>> 1970-01-01T01:00:01+0100 */], 0) = 0
>> chmod("/gnu/store/whhc80cx1zpi9cj3bil3sp8wwzcf8p9p-image-root/gnu/store/n6l40ygclxfysnc2nxwrslnn5ab6wv3v-profile/lib/modules/5.4.47-gnu/kernel/net/netfilter",
>>  040555) = 0
>> getrusage(RUSAGE_SELF, {ru_utime={tv_sec=1306, tv_usec=334062}, 
>> ru_stime={tv_sec=143, tv_usec=819675}, ...}) = 0
>> clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {tv_sec=1450, tv_nsec=153748295}) = 
>> 0 
>> lstat("/gnu/store/whhc80cx1zpi9cj3bil3sp8wwzcf8p9p-image-root/gnu/store/n6l40ygclxfysnc2nxwrslnn5ab6wv3v-profile/lib/modules/5.4.47-gnu/kernel/net/netfilter/xt_connlimit.ko",
>>  {st_mode=S_IFLNK|0777, st_size=125, ...}) = 0
> As discussed on IRC, those "image-root" derivations do exercise store
> deduplication a lot, are could participate to the observed database
> contention.
> Turns out I used those derivations as "staging" directories that are
> passed to GNU Xorriso when producing ISO images and "mke2fs" and
> "mkdosfs" when producing raw disk-images. Using the store was also
> convenient to ensure root ownership of the image files.
> With 1cb9effc3e875ec7bae31bb06069d16ac89f7e1d and
> 7f75a7ec08975eb6d6e01db61bd6b91f447f655e, I remove those "image-root"
> derivations, and proceed to image creation in a single derivation.

Nice, thanks for the prompt change!

I’ve been monitoring berlin a bit and I’ve become quite convinced that
these big directories are what triggered contention.  The good thing is
that this has allowed us to find weaknesses in (guix store database) and
(guix store deduplicate).

> The later commit is a bit more tricky because "mke2fs" is not able to
> override file permissions. I used "fakeroot" to make sure that the raw
> disk-image files are owned by "root" and not "guixbuilder".

OK, makes sense.

Maybe at some point we’ll have to use libext2fs directly or patch mke2fs
to get the desired flexibility.

Thank you!


reply via email to

[Prev in Thread] Current Thread [Next in Thread]