--- Begin Message ---
|
Subject: |
Disarchive as a fallback for downloads |
|
Date: |
Tue, 23 Mar 2021 00:42:12 -0400 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) |
Hello,
This patch series adds Disarchive assembly (backed by SWH lookup) as a
fallback for downloads.
To try it, make sure you are running the daemon in an environment with
Disarchive available:
$ ./pre-inst-env guix environment --ad-hoc guile disarchive
# ./pre-inst-env guix-daemon --build-users-group=guixbuild
Don’t forget to stop your existing Guix Daemon. :)
You also need to make sure that regular downloads are unavailable. I do
this by adjusting the “try” loop at the end of “url-fetch” in
“guix/build/download.scm”. I replace the usual list of URLs with ‘()’:
(let try ((uri (append uri content-addressed-uris)))
(match '() ; uri
...))
Now you can ask Guix for a recent .tar.gz source package:
$ ./pre-inst-env guix build --no-substitutes -S python-httpretty
You should see:
Trying to use Disarchive to assemble
/gnu/store/kbcnm57y2q1jvhvd8zw1g5vdiwlv19y9-httpretty-1.0.5.tar.gz
Assembling the directory httpretty-1.0.5
Downloading from Software Heritage...
7903d608efc89c14afb4d692a3721156e31a43e2/
7903d608efc89c14afb4d692a3721156e31a43e2/httpretty-1.0.5/
7903d608efc89c14afb4d692a3721156e31a43e2/httpretty-1.0.5/COPYING
[...]
Checking httpretty-1.0.5 digest... ok
Assembling the tarball httpretty-1.0.5.tar
Checking httpretty-1.0.5.tar digest... ok
Assembling the Gzip file httpretty-1.0.5.tar.gz
Checking httpretty-1.0.5.tar.gz digest... ok
Copying result to
/gnu/store/kbcnm57y2q1jvhvd8zw1g5vdiwlv19y9-httpretty-1.0.5.tar.gz
successfully built
/gnu/store/k0b3c7kgzyn1nlyhx192pcbcgbfnhnwa-httpretty-1.0.5.tar.gz.drv
There’s lots to talk about though....
First, it looks up the metadata on my server. This is fine for a demo,
but not what we want forever. The patch series supports adding several
mirrors for looking up the metadata. In the past, we talked about
putting everything on one or a few of the big Git hosting platforms like
GitHub or Gitlab. That way, it would be easily picked up by SWH and
archived “forever”. Right now, I have Cuirass set up to build the
metadata, and a little script that moves it from the build server to my
Web server. It would be simple enough to adjust that script to push it
to a remote Git repo. (Of course, the next step is to move this setup
to Guix infrastructure.) Thoughts?
On the code level, there were two things I couldn’t figure out for
myself.
I made the mirror list just simple strings. AIUI, the client and the
daemon have to agree about the format of the mirror list. Given that
running old daemons is common, changing the format is difficult. Is it
worth it to copy the more flexible interface used by the content
addressed mirrors? If yes, do I have to do the same ‘module-autoload!’
dance to use ‘bytevector->base16-string’? :) (I probably would have
just copied it, but that part confused me a bit.)
I imported some modules from “guix/build/download.scm” (well, just
“base16” and “swh”). It feels weird to use a bunch of host-side modules
from what’s nominally a “guix/build” module. This is okay because
“guix/build/download.scm” is not /really/ build-side code. It’s more
like daemon (-ish) code that just happens to live in “guix/build”, which
is why importing host-side modules is OK... right?
Hopefully everything else is more-or-less fine. :)
-- Tim
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#47336: Disarchive as a fallback for downloads |
|
Date: |
Thu, 29 Apr 2021 13:24:09 -0400 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hello,
Ludovic Courtès <ludo@gnu.org> writes:
> Timothy Sample <samplet@ngyro.com> skribis:
>
>> And broke “guix pull”!! (I somehow fooled myself into thinking that I
>> had already tested with “guix pull --url=...” locally.) I reverted the
>> offending commit.
>
> You can test with ‘guix pull’ (you need to make sure to specify the
> right file:// URL *and* branch), or you can run “make as-derivation”.
I will definitely be more careful with this in the future.
>> [...] Does this approach look okay?
>
> That’s one possibility.
>
> The patch below takes another approach. I think it aesthetically
> slightly more pleasant because we don’t have to play ‘resolve-module’
> tricks for obscure reasons. WDYT?
This is exactly what I was hoping for, but I couldn’t quite connect all
the dots in “build-self.scm”. Thanks!
> (It also fixes a format string argument mismatch.)
Good catch!
I’ve pushed the updated patch and am closing the issue. :)
-- Tim
--- End Message ---