Re: Downloader for "wrapped" tarbar?

[Marius Bakke]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Hi,
>
> was just written in another mail, I'm currently working on a
> erlang/rebar build system. This includes an importer from hex.pm, a
> package repository for elixir and erlang packages. (Since this is build
> into rebar3 I assume it what PyPI is for Python and CPAN for Perl.)
>
> At hex.pm, packages are provided in a tarfile [1] wrapping the source
> tar-file:
>
> -rw-r--r-- 0/0               1 2017-06-14 21:57 VERSION
> -rw-r--r-- 0/0              64 2017-06-14 21:57 CHECKSUM
> -rw-r--r-- 0/0             532 2017-06-14 21:57 metadata.config
> -rw-r--r-- 0/0            4744 2017-06-14 21:57 contents.tar.gz
>
> IMHO it does not make sense to keep this wrapping tar-file in the store.
>
> So my idea is to create a "hexpm-fetch" method, which downloads the
> tar-file and only stores the "content.tar.gz" in the store (using a
> proper name, of course).
>
> How can this be done?

Tarballs from rubygems.org has the same problem and works around it by
special support in ruby-build-system.

It would be ideal to have an origin method that could extract the
"inner" tarball, i.e. contents.tar.gz for hex.pm and data.tar.gz in the
case of RubyGems.  As zimoun mentioned, a good place to start is look at
how other origin methods are implemented such as url-fetch/tarbomb, etc.

signature.asc
Description: PGP signature