Why linux-libre source code is not in sources.json

[zimoun]

Hi,

On Thu, 09 Sep 2021 at 00:00, Ludovic Courtès <ludo@gnu.org> wrote:

> Now we have to see what’s available on berlin & co., and the extent to
> which SWH can help with this situation.  So far I’m not sure about the
> tarball contents, but the repo seems to be archived:

As we know, the issue with tarball on SWH is that we cannot guarantee
the map between the information stored at package time and the
information SWH serves at fetch time.  Metadata, etc. so SWH cooking can
return a tarball having the checksum Guix that expects or something
different.

We (at least me :-)) need to invest energy into Disarchive.
Definitively! 

> For some reason, we seem to be exporting only one tarball in
> sources.json currently (the file that SWH periodically reads):
>
> --8<---------------cut here---------------start------------->8---
> $ wget -qO - https://guix.gnu.org/sources.json|jq |grep linux-libre.fsfla.org
>         
> "https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz";,
>         
> "https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz";,
> --8<---------------cut here---------------end--------------->8---
>
> We should check why we’re not providing all the URLs and fix it.

Interesting… :-)

The file ’https://guix.gnu.org/sources.json’ is basically built using
’fold-packages’ (see ’all-packages’ in
guix-artwork/website/apps/packages/data.scm) and ’package-source’ (see
’sources-json-builder’ in
guix-artwork/website/apps/packages/builder.scm).  And indeed,
’linux-libre’ is missing.

Let consider this snippet as ’/tmp/foo.scm’ mimicking the builder of
JSON files:

--8<---------------cut here---------------start------------->8---
(use-modules (guix packages)
             (gnu packages))

(define all-packages
  (sort
   (fold-packages (lambda (package lst)
                    (if (or (package-superseded package)
                            (package-replacement package))
                        lst
                        (cons package lst)))
                  '())
   (lambda (p1 p2)
     (string<? (package-name p1)
               (package-name p2)))))

(map (lambda (p)
       (format #t "~a~%" (package-source p)))
     all-packages)
--8<---------------cut here---------------end--------------->8---

Then indeed, only 5.4.20 is seen.

--8<---------------cut here---------------start------------->8---
$ guix repl /tmp/foo.scm | grep 'linux-libre.fsfla'

#<origin 
("https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz";
 "ftp://alpha.gnu.org/gnu/guix/mirror/linux-libre-5.4.20-gnu.tar.xz"; 
"mirror://gnu/linux-libre/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz") 
#<content-hash sha256:1qxhf6dmcwjblzx8fgn6vr10p38xw10iwh6d1y1v1mxb25y30b47> () 
7faf439cfba0>
#<origin 
("https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz";
 "ftp://alpha.gnu.org/gnu/guix/mirror/linux-libre-5.4.20-gnu.tar.xz"; 
"mirror://gnu/linux-libre/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz") 
#<content-hash sha256:1qxhf6dmcwjblzx8fgn6vr10p38xw10iwh6d1y1v1mxb25y30b47> () 
7faf439cfba0>
--8<---------------cut here---------------end--------------->8---

The difference is that this package ’linux-libre-headers-5.4.20’ is
created with ’make-linux-libre-headers’ and the others with
’make-linux-libre-headers*’.  Subtle. ;-)  In other words,

--8<---------------cut here---------------start------------->8---
$ guix repl
scheme@(guix-user)> ,use(guix packages)
scheme@(guix-user)> ,use(gnu packages linux)
scheme@(guix-user)> (package-source linux-libre-headers-5.4.20)
$1 = #<origin 
("https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz";
 "ftp://alpha.gnu.org/gnu/guix/mirror/linux-libre-5.4.20-gnu.tar.xz"; 
"mirror://gnu/linux-libre/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz") 
#<content-hash sha256:1qxhf6dmcwjblzx8fgn6vr10p38xw10iwh6d1y1v1mxb25y30b47> () 
7f881c1ec4e0>
scheme@(guix-user)> (package-source linux-libre-headers-5.13)
$2 = #<origin #<promise #<procedure 7f881c1dd900 at 
gnu/packages/linux.scm:248:8 ()>> #<content-hash sha256:#f> (#<origin 
"http://www.fsfla.org/svn/fsfla/software/linux-libre/lemote/gnewsense/branches/3.16/100gnu+freedo.patch";
 #<content-hash sha256:1hk9swxxc80bmn2zd2qr5ccrjrk28xkypwhl4z0qx4hbivj7qm06> () 
7f881c1ec840> #<origin 
"https://salsa.debian.org/kernel-team/linux/raw/34a7d9011fcfcfa38b68282fd2b1a8797e6834f0/debian/patches/bugfix/arm/arm-mm-export-__sync_icache_dcache-for-xen-privcmd.patch";
 #<content-hash sha256:1ifnfhpakzffn4b8n7x7w5cps9mzjxlkcfz9zqak2vaw8nzvl39f> () 
7f881c1ec7e0> 
"/gnu/store/yrqr7syxbm4pddzlgc4pwn9wixmpy9xh-guix-module-union/share/guile/site/3.0/gnu/packages/patches/linux-libre-arm64-generic-pinebook-lcd.patch")
 7f881c1ec780>
--8<---------------cut here---------------end--------------->8---

Therefore, the builder of JSON (mainly ’origin->json’) does not consider
such cases and assume that ’origin-uri’ can be applied.  Well, I will
try to improve the situation if no one beats me. :-)


Cheers,
simon