Re: Inverted index to accelerate guix package search

[Giovanni Biscuolo]

Hi Simon and all other developers

first and foremost: I'm sorry I still cannot help in developing this
"xapian feature" or even explore pros and cons of this feature

for now mine it's just a user and possible beta-tester POV... let's call
it food for thought :-)

zimoun <address@hidden> writes:

[...]

>> Actually I would love to search (tag?!?) for packages/services/machines
>> (profiles?) the same way I query my email database with notmuch: it
>> would give me a fourth dimension in "DevOps" :-)
>
> Currently, you can output all the packages with 'guix search ""' and
> index them using the Python API. It is not so nice but it should give
> an idea about what to expose first to Guile.

interesting approach for an indexer external to Guix, let's call it a
proof of concept...

I'd really like to experiment this directly in Guile but I'm not able to
help with Guile bindings

>> I cannot find Guile in Xapian bindings [1] but NotDeft [2] have some
>> Emacs Lisp code to interact with Xapian
>
> There is some work. IMHO.

for sure :-) this is why i defined this low-pri in the general Guix
plans

> If Guix goes to Xapian, all the Git history of all the packages should
> be indexed. Today, it is really hard to find the right commit
> corresponding to the right version -- the only solution I have is "git
> log"+grep; not really friendly.

indexing the all the history could be very interesting, but it's enough
interesting to me having a system to query (and tag if needed) Guix info
from a point in time, i.e. index creation

thinking about implementation, IMHO indexing the output of "guix search"
is doable but indexing the commit logs of git is complex, probably too
much complex

> Some discussion spoke about using an external index, fetched for
> example on the Guix Data Service. But I personally do not like to rely
> on external service accessible through the network.

me too, as I said I'd like something like notmuch - guixmuch :-) - that
indexes my mailbox; in Guix terms I see "my mailbox" [1] as
packages/services/machines and all other useful metadata I have in all
my channels/profiles

...then I have muchsync to sync the notmuch database across different
machines, and a similar feature whould be nice for guixmuch :-O


[1] where IMAP is replaced by git (in various repos, for
packages/config/channels) and offlineimap is replaced by guix pull

> However, note that index all the packages of all the Guix history
> using guile-git+fold-packages is not straightforward: resource
> consuming and piece of well-thought data structures.
>
> Well, now "guix time-machine" is here, Guix is lacking tools to browse
> the history of all the packages.

we are asking too much to Guix, but it is an interesting feature

Thanks! Gio'

[...]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

signature.asc
Description: PGP signature