Re: Linux-libre 5.8 and beyond

[Alexandre Oliva]

Hello, Mark,

On Aug 15, 2020, Mark H Weaver <mhw@netris.org> wrote:

> I was talking about my hope to enable users, *on their own
> machines* and using *their own private build recipes*, to make a
> best-effort deblobbing of a non-standard kernel variant that they need
> to use for whatever reason.

A non-free kernel, standard or not, shouldn't really be in scope for a
FSDG distro, IMHO.  Even the pointer to the non-Free releases used as a
starting point for build recipes comes across as undesirable to me, more
so when there's an expectation (and such a high concern) for enabling
users to use them, with a near-certainty that this will likely go
silently wrong freedom-wise.


> If they aren't provided with that option,
> the obvious alternative (which I expect 99% of such users would do
> anyway) is to simply run a fully-blobbed kernel instead.

I'm surprised that they'd prefer to run deblobbing and checking at each
point of a bisection, over applying the deblobbing changes as a patch,
or even starting from a Free release, rebasing a set of changes to test
onto it, and quickly building and bisecting that.  That's what I would
rather do.  But then, I probably wouldn't be using the guix build recipe
and default kernel config for the bisection, but rather a smaller config
built within the bisect tree.


> Alexandre Oliva <lxoliva@fsfla.org> wrote:

>> I'm sure that's not what you intend, but this arrangement, plus your
>> mention of hurriedly getting releases out, adds up to an incentive to
>> disable the deblobbing so as to get a faster build.

> I don't understand how you reached this conclusion.  As far as I can
> tell, changing Guix to run the deblob scripts made *no* difference to
> what someone would have to do to ask Guix to build fully-blobbed
> kernel.

One of the issues, as you'd pointed out, was time pressure to get a
build completed.  If someone is under such pressure, and knows that
deblobbing will take 30 minutes, and that verifying the deblobbed tree
will take another 30 minutes (or 24 hours, if using the wrong tool for
the job), one might disable the cleaning up rather than figuring out how
to get the recipe to use an already cleaned and verified release.




> In particular, if I can easily run an
> automated process on my own machine instead of relying on some other
> system to provide pre-generated outputs for me, then I prefer to do it
> myself.

That's at odds with the time pressure you mentioned before.


Now, let me get something straight.  You seem to have got the idea that
I oppose verification of our releases.  That's very very wrong.  I
welcome verification.  I just don't see that this belongs in the guix
build process.


I get it that guix packages several projects that need cleaning up.
IMHO, guix build recipes should NOT point at such upstream projects
along with the cleaning up recipes.  This should be part of a separate
recipe, namely, that of packaging/verifying/blessing *sources* for use
in guix.  Once the sources are packaged (in a verifiable/reproducible
way), they should be made available by the distro to users.  These are
the corresponding sources that we expect every distro to offer.

It's not just about builders getting those sources, verifying them (or
not) and making binaries out of them.  Any user ought to be entitled to
request corresponding sources to binaries provided by guix, and guix
should be able to provide them without requiring users to run
potentially complex procedures that might even end up producing
different results, depending on platform-specific bugs, versions of
tools, not to mention the various other potential sources of
non-reproducible sources and binaries.  

Even if the procedures are meant to be reproducible, you'll only know
they aren't when you manage to trace a difference in a packaged binary
back to a difference in sources, when you can no longer reproduce the
sources used before.  Archiving the sources proper, for verification and
for distribution to users as corresponding sources, would avoid
surprises of non-reproducible procedures being found out long after the
fact, just when corresponding sources are requested and can't be
provided any more.

I'm ambivalent as to whether patches that guix wishes to apply should be
applied as part of source packaging, or have the patches made available
separately.  I can see arguments both ways.  On the one hand, applying
patches, as reproducible as it normally is, might be subject to
occasional variations, especially when the line numbers or the contexts
in the patch are inexact.  On the other, these cases are extremely rare,
and being able to reuse a base tarball while trying out some patches,
without having to repackage a base tarball, and having patches
conspicuously presented to builders and users, separately from an
upstream base release, is desirable when the patches are not meant to
address freedom issues (those that do address freedom issues had better
be applied by other means during source packaging, to avoid publishing
reversible patches that could be used to reintroduce the freedom
issues).  This suggests there could be support for patches in both
source preparation recipes, and in build recipes.

For projects that need cleaning up, the source packaging recipe could
apply any needed cleaning up.  For projects like GNU Linux-libre, that
are already cleaned up, or most other packages that don't need any
cleaning up whatsoever, the source preparation recipe could be as simple
as downloading the sources, as well as any signatures thereof, checking
that they match, and recording the checksums of the sources to be used
for binary building.

Source preparation might also offer a verify mode, that would *also*
fetch the sources from a corresponding release of the project that needs
cleaning up, perform the cleaning up and compare the results, but I'd
much rather links to the corresponding projects that need cleaning up be
pushed out of FSDG-compliant distros.  Maintainers of such packages
could and probably should run such verification themselves, without
exposing every builder to the non-Free pointers and code.

I urge guix to address the problem of build recipes pointing to non-Free
packages and getting builders to download non-Free Software onto their
machines.

It would probably be wise to discuss more broadly how FSDG distros can
document and share their cleaning up procedures, so that builders and
users can double-check them if they wish to, and so that other FSDG
distros can cooperate and reuse.  Clearly we don't wish distro
maintainers to keep these private to themselves, but we surely don't
want links to sources containing unacceptable sources to be conspicuous
in the distro either, let alone being used when Free sources are or
could be readily available.


Now, I wonder...  If sources for projects other than Linux-libre and
GNUzilla need cleaning up, perhaps it would make sense for our
community, e.g. GNU, to undertake the source cleaning up, releasing
clean sources for all interested distros and users to get to.  Perhaps
we could encourage maintainers of the such packages in the various Free
distros to share and divide the workload of maintaining them and the
cleaning up recipes, for everyone's benefit.  Then guix could just point
at the clean sources released by this project, instead of going through
the significant change of introducing separate 'prepare sources' recipes
to avoid pointing users at non-Free sources through build recipes.


-- 
Alexandre Oliva, happy hacker
https://FSFLA.org/blogs/lxo/
Free Software Activist
GNU Toolchain Engineer