[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiti
From: |
Maxime Devos |
Subject: |
[bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting |
Date: |
Sat, 05 Mar 2022 10:44:48 +0100 |
User-agent: |
Evolution 3.38.3-1 |
Ludovic Courtès schreef op vr 04-03-2022 om 21:45 [+0100]:
> My concern is that we’d end up caching one or two little files in
> ~/.cache for each candidate package, and (rate limit aside) the overhead
> of dealing with the cache might outweigh the benefits. I’d rather use
> ‘http-fetch/cached’ for bigger files, like in (guix cve).
>
> WDYT?
If the overhead of caching little files is a concern, then perhaps a
SQLite (or GDBM) database could be used instead of the filesystem-based
cache? The number of packages in Guix was about 150 000 IIRC, if we
assume something around the magnitude of 200 bytes per package, then
we end up with about 29 MiB for the entirity of Guix. And there might
be some opportunities for compression, reducing this number.
Something like this could be left for later though.
Greetings,
Maxime.
signature.asc
Description: This is a digitally signed message part
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, (continued)
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Ludovic Courtès, 2022/03/05
- [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion., Maxime Devos, 2022/03/05
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Ludovic Courtès, 2022/03/05
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Maxime Devos, 2022/03/05
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Ludovic Courtès, 2022/03/06
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Maxime Devos, 2022/03/05
- [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion., Maxime Devos, 2022/03/05
- [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Ludovic Courtès, 2022/03/05
[bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting, Maxime Devos, 2022/03/04