--- Begin Message ---
|
Subject: |
Disarchive as a fallback for downloads |
|
Date: |
Tue, 23 Mar 2021 00:42:12 -0400 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) |
Hello,
This patch series adds Disarchive assembly (backed by SWH lookup) as a
fallback for downloads.
To try it, make sure you are running the daemon in an environment with
Disarchive available:
$ ./pre-inst-env guix environment --ad-hoc guile disarchive
# ./pre-inst-env guix-daemon --build-users-group=guixbuild
Don’t forget to stop your existing Guix Daemon. :)
You also need to make sure that regular downloads are unavailable. I do
this by adjusting the “try” loop at the end of “url-fetch” in
“guix/build/download.scm”. I replace the usual list of URLs with ‘()’:
(let try ((uri (append uri content-addressed-uris)))
(match '() ; uri
...))
Now you can ask Guix for a recent .tar.gz source package:
$ ./pre-inst-env guix build --no-substitutes -S python-httpretty
You should see:
Trying to use Disarchive to assemble
/gnu/store/kbcnm57y2q1jvhvd8zw1g5vdiwlv19y9-httpretty-1.0.5.tar.gz
Assembling the directory httpretty-1.0.5
Downloading from Software Heritage...
7903d608efc89c14afb4d692a3721156e31a43e2/
7903d608efc89c14afb4d692a3721156e31a43e2/httpretty-1.0.5/
7903d608efc89c14afb4d692a3721156e31a43e2/httpretty-1.0.5/COPYING
[...]
Checking httpretty-1.0.5 digest... ok
Assembling the tarball httpretty-1.0.5.tar
Checking httpretty-1.0.5.tar digest... ok
Assembling the Gzip file httpretty-1.0.5.tar.gz
Checking httpretty-1.0.5.tar.gz digest... ok
Copying result to
/gnu/store/kbcnm57y2q1jvhvd8zw1g5vdiwlv19y9-httpretty-1.0.5.tar.gz
successfully built
/gnu/store/k0b3c7kgzyn1nlyhx192pcbcgbfnhnwa-httpretty-1.0.5.tar.gz.drv
There’s lots to talk about though....
First, it looks up the metadata on my server. This is fine for a demo,
but not what we want forever. The patch series supports adding several
mirrors for looking up the metadata. In the past, we talked about
putting everything on one or a few of the big Git hosting platforms like
GitHub or Gitlab. That way, it would be easily picked up by SWH and
archived “forever”. Right now, I have Cuirass set up to build the
metadata, and a little script that moves it from the build server to my
Web server. It would be simple enough to adjust that script to push it
to a remote Git repo. (Of course, the next step is to move this setup
to Guix infrastructure.) Thoughts?
On the code level, there were two things I couldn’t figure out for
myself.
I made the mirror list just simple strings. AIUI, the client and the
daemon have to agree about the format of the mirror list. Given that
running old daemons is common, changing the format is difficult. Is it
worth it to copy the more flexible interface used by the content
addressed mirrors? If yes, do I have to do the same ‘module-autoload!’
dance to use ‘bytevector->base16-string’? :) (I probably would have
just copied it, but that part confused me a bit.)
I imported some modules from “guix/build/download.scm” (well, just
“base16” and “swh”). It feels weird to use a bunch of host-side modules
from what’s nominally a “guix/build” module. This is okay because
“guix/build/download.scm” is not /really/ build-side code. It’s more
like daemon (-ish) code that just happens to live in “guix/build”, which
is why importing host-side modules is OK... right?
Hopefully everything else is more-or-less fine. :)
-- Tim
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#47336: Disarchive as a fallback for downloads |
|
Date: |
Tue, 27 Apr 2021 22:30:39 -0400 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hi,
Ludovic Courtès <ludo@gnu.org> writes:
> Ping²!
>
> Let me know if you’re like me to apply the patches on your behalf.
No, no. I’m just a little distracted over here. I just pushed this
series with the updates you suggested (using #:autoload, passing
#:verify-certificates?, and being a bit more chatty). Sorry for the
delay and thanks for the reminder.
Next, I’ll convert my Cuirass 0.x setup to a Cuirass 1.x setup, and then
I can start a discussion about moving the metadata builds to
ci.guix.gnu.org.
Also, to answer your other question:
> So we would normally arrange so that the ‘guix’ package depends on
> Disarchive, such that the above ‘resolve-module’ call works when done
> via ‘guix perform-download’, right?
That’s the idea. I’m not confident about updating the ‘guix’ package
myself, though....
-- Tim
--- End Message ---