(Re-) Designing extractong-downaloder

[Hartmut Goebel]

Hi,

TL;DR: What do you think about the idea of an „extracting dowloader“?

I'm about pick up work on „extracting downloader“ and the rebar build 
system (for erlang), see <https://issues.guix.gnu.org/51061> for a first 
try, In the aforementioned issue some points came up regarding the basic 
design of the patch. Thus before starting to write code, I'd like to 
agree on a basic design.

The basic idea behind „extracting downloader“ is as follows: Packages 
provided by hex.pm (the distribution repository for erlang and elixir 
packages) are tar-archives containing some meta-data files and the 
actual source (contents.tar.gz), see example below, So the ideas was to 
only store the contents.tar.gz (instead of requiring an additional 
unpacking step).

In some earlier discussion someone mentioned, this could be interesting 
for ruby gems, too.

Storing only the archive would allow to have the archive's hash as the 
"source"-hash and allow for easy validation of the hash. Anyhow, much of 
the complexity of the current implementation (see issue 51061) is caused 
by this idea, since the code needs to postbone hashing to after the 
download.

Also In some earlier discussion Ludo (afair) brought up the point 
whether e.g. swh would be able provide a source-package if hased this way.

What do you think about the idea of an „extracting dowloader“?


Example for a package from hex.pm:

$ wget https://repo.hex.pm/tarballs/getopt-1.0.2.tar
…
$ tar tvf getopt-1.0.2.tar
-rw-r--r-- 0/0               1 2000-01-01 01:00 VERSION
-rw-r--r-- 0/0              64 2000-01-01 01:00 CHECKSUM
-rw-r--r-- 0/0             451 2000-01-01 01:00 metadata.config
-rw-r--r-- 0/0           14513 2000-01-01 01:00 contents.tar.gz


--
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |