Re: [Dragora-users] Distribution of ARM rootfs tarballs

[Kevin "The Nuclear" Bloom]

Matias Fonzo writes:

> El 2020-01-31 00:06, Kevin "The Nuclear" Bloom escribió:
>> Thanks for the quick reply, Matias. See my comments below:
>>
>>> El 2020-01-29 16:50, Kevin "The Nuclear" Bloom escribió:
>>>> Hi,
>>>
>>> Hello Kevin.  :-)
>>>
>>>> Those of us who have a C201 know that installation on this device is
>>>> quite nontraditional. Instead of booting off of a USB stick and running
>>>> an installer, one must do it manually by loading an sd card (or usb
>>>> stick) with a special kernel partition and a special root
>>>> partition. What this means is that creating an ISO for this machine is
>>>> pointless. Due to that, most distros that support the machine have a
>>>> rootfs tarball that you unpack into the root partition and, normally,
>>>> inside of /boot there is a linux.kpart or something that gets written to
>>>> the kernel partition using `dd`.
>>>
>>> Okay.  Question: what format would be appropriate for create the rootfs?.
>>>
>>
>> Arch-arm uses tar.gz and we probably should stick to that because some
>> people might be unpacking it from ChromeOS which doesn't come with lzip
>> installed. It can, however, unpack gzip.
>>
>>>> That being said, I'm curious as to how we wish to handle the
>>>> distribution of Dragora 3 rootfs tarballs for this machine. Most
>>>> distros' tarball is quite small and only contains the core system with
>>>> simple network tools such as wpa-supplicant for connecting the machine
>>>> to the internet (there is no Ethernet port, so wpa will be
>>>> required). Once the core system is booted the user is expected to
>>>> install the rest of the system via their package manager. Since Dragora
>>>> doesn't have a package repo that contains precompiled binaries (that I'm
>>>> aware of), I'm not sure how we want to do this.
>>>
>>> Here we could say that Dragora's "kernel" includes everything needed to boot
>>> the
>>> system, as well as the network part, including the wpa_supplicant currently.
>>> As
>>> for the packages, we can say that the official packages are provided and
>>> distributed after each release[1].  In this sense, it is not a high priority
>>> (for me) to provide updates to pre-compiled packages like any other
>>> pre-compiled
>>> package, since the distribution has to be finished, or at least until it
>>> reaches
>>> the stable one.
>>>
>>> [1] http://rsync.dragora.org/v3/packages/
>>>
>>
>> I think that is a good idea. Would take the stress away from trying to
>> keep every package up-to-date all the time. I'm still curious about how
>> we should manage downloading the binaries and then installing them in
>> the correct order. Any ideas how to do this? (i.e. `wget -i
>> BINARY-LIST.txt | qi -i` or something)
>
> Qi can read from standard input, for example if the file currently contains 
> the
> full (local) path of one or more packages, it can install them, e.g: qi -i - <
> pkglist.txt
>
> What you want is to read, download and install.  Currently Qi has the code to
> download and generate the .sha256 on the source side.  As a pending issue, we
> could use or adapt this code (as it declares the General Network Downloader) 
> to
> tell Qi to download the packages when using the -i option and if "http(s)://" 
> is
> specified on the command line.
>
> Of course, this has to be studied to make it as reliable as possible (.sha256,
> signatures...).
>

That would be quite handy! If this would be valuable to the other D3
archs then I think it would be great addition, otherwise, we may want to
just have a shell script that does this using wget+qi or
something.

>>>> My idea is this: we do the same thing that other distros do, for the
>>>> most part. Keep the tarball small and use just the core system with some
>>>> networking programs. The kernel will be in /boot under a name like
>>>> kernel.kpart or something. Inside of the root home directory there will
>>>> be a few different text files that contain urls to pre-compiled binary
>>>> packages. Each file will have names that match up with the .order files
>>>> when building D3: editors.txt, sound.txt, xorg.txt, etc. They will have
>>>> all the programs in the orders that they need to be in to insure a safe
>>>> installation. Then, the user uses a few commands to download and install
>>>> each package (probably something with wget that passes the binary into a
>>>> qi command). Once they've installed all the stuff they need, they'll be
>>>> good to go!
>>>
>>> What I see here is that it is possible that the kernel configuration needs 
>>> to
>>> be
>>> adjusted[2], in addition to testing it (very important), I do not own such a
>>> computer, and if I did, I would not have enough time now to focus 
>>> exclusively
>>> on
>>> this, considering all that needs to be done. I keep thinking about how these
>>> lists will facilitate the installation of the packages (how to produce them
>>> from
>>> Qi), for the moment you can compile the core[3] and produce the rootfs, then
>>> compile the rest to get the packages...
>>>
>>> [2]
>>> http://git.savannah.nongnu.org/cgit/dragora.git/plain/archive/kernel/config-c201-v7
>>> [3]
>>> http://git.savannah.nongnu.org/cgit/dragora.git/plain/recipes/00-core.order
>>>
>>
>> Yes, I just completed the core build with the current master
>> branch. Everything went smoothly except for meson, which has always been
>> a problem child on the C201. I will be creating the signed kernel and
>> attempting booting tomorrow, if time permits.
>>
>>>> Let me know if this is a good idea or if it need tweaked at all! This is
>>>> quite a lot of work for only 1 machine but it's the only way I can think
>>>> of other than just having all that stuff in the tarball but that would
>>>> make it very large.
>>>
>>> I will try to assist you and provide you with what you need.
>
> What I can think of is that we can create a new scenario for the bootstrapping
> process.  This would be a minimal system to boot and log in to, from there you
> could install whatever you want, reusing the minimal system tools.   This will
> allow you:
>
> - Check and test the kernel configuration.
> - Save time instead of building the stage1, the whole core, etc.
> - Accessible via enter-chroot.
> - Have the rootfs small.
> - Ready to boot.
>
> For example, you would set the cross compiler in motion:
>
> ./boostrap -s0 -a armv7_hf
>
> Then you would produce the minimum system using the cross compiler for your
> machine:
>
> ./bootstrap -s201 -a armv7_hf
>
> If you already have the cross-compiler in place, you would use the "201"
> scenario/stage as many times as necessary (related changes, kernel
> configuration, busybox, settings, etc.)
>
> In time, the new produced rootfs will be adjusted to what is "just and
> necessary".
>
> ... Cum on feel the noize! ;-)
>

This would be great! Would it be possible to just have another special
.order file called `base.order` or `minimal.order`? Which would just
build the essentials and some network stuff. Using the bootstrap command
you mention would work too!

>>>> Thanks,
>>>>
>>>> Kev