[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Faster “guix search” (was Re: How many bytes do we add (closure of guix)
From: |
Simon Tournier |
Subject: |
Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) |
Date: |
Wed, 31 May 2023 10:05:15 +0200 |
Hi,
On Tue, 30 May 2023 at 21:10, Csepp <raingloom@riseup.net> wrote:
> It makes zero sense to load full package definitions from
> disk for most queries, such as guix search, with an SoA representation
> we could load only the fields that we care about.
That’s already the case; see
~/.config/guix/current/lib/guix/package.cache.
For instance, “guix package -A” exploits it and the performances are
acceptable. Two past summers, wow already! I tried to augment it and
exploit it for “guix search”. The implementation and benchmark is in
#39258 [1]. Well, the whole thread of #39258 appears to me worth to
consider because it spots various bottleneck specific to “guix search”
and explains why the improvement is not straightforward.
Well, I have started months ago to write a Guix extension using
guile-xapian. My aim is to tackle two annoyances: 1. the speed and
2. the relevance.
About the relevance #2, the issue is that the current scoring considers
only the local information of one package without considering the global
information of all the others. Well, see [2,3,4] for some details. :-)
1: https://issues.guix.gnu.org/39258#119
2:
https://yhetil.org/guix/CAJ3okZ3E3bhZ5pROZS68wEKdKOcZ8SpXsvdi-bnB=9Jz3mPahA@mail.gmail.com
3:
https://yhetil.org/guix/CAJ3okZ3+hn0nJP98OhnZYLWJvhLGpdTUK+jB0hoM5JArQxO=zw@mail.gmail.com
4:
https://yhetil.org/guix/CAJ3okZ0LaJzWDBA7bjqZew_jAmtt1rj9PJhevwrtBiA_COXENg@mail.gmail.com
> ps.: Now I'm even more glad that I'm using a file system with
> transparent compression on all my Guix systems.
Did you benchmarked the performances for some Guix operations on these
compressed vs uncompressed file system?
Cheers,
simon