[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Avoiding the garbage collector for the Guix Data Service
From: |
Christopher Baines |
Subject: |
Re: Avoiding the garbage collector for the Guix Data Service |
Date: |
Wed, 12 Feb 2020 23:54:36 +0000 |
User-agent: |
mu4e 1.2.0; emacs 26.3 |
Ludovic Courtès <address@hidden> writes:
> Hi Christopher,
>
> Christopher Baines <address@hidden> skribis:
>
>> The Guix Data Service has Guix compute lots of derivations, and then it
>> loads the data in to a database. I'm having some issues with the garbage
>> collector though, as sometimes it removes data from the disk before I've
>> had a chance to read it (at least I think this is what's happening, I
>> think this job failed for this reason [1]).
>>
>> 1: http://data.guix.gnu.org/job/10638
>>
>> I'm not quite sure what the best strategy is to combat this? I had a
>> look, and I couldn't see how to acquire a lock to prevent the garbage
>> collector from running. I thought about registering thousands of gc
>> roots, but that seems to just move the race a bit, as there's still a
>> period between generating the derivation and registering the root where
>> it can be deleted.
>>
>> Any ideas?
>
> At the Guix Days we discussed using ‘add-temp-root’ and similar to
> ensure that .drv files are at least GC-protected for the duration of the
> session (connection to the daemon).
>
> Does that address your problem?
I think so, it at least seems to greatly reduce the chance the
derivations get removed. I made the Guix Data Service change here [1],
and I haven't seen the issue since.
1:
https://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=f59354ed23a9c50e43b7aaaeaddb5365feda4e29
Thanks,
Chris
signature.asc
Description: PGP signature