[bug#38390] [core-updates] Scheme-only bootstrap: merge wip-bootstrap

[Ludovic Courtès]

Hello Janneke & Timothy!

Jan Nieuwenhuizen <address@hidden> skribis:

> Gash has now reached pretty good posix compliance, which really helps to
> run configure scripts and sh snippets in autotool-generated Makefiles.
>
> Gash Core Utils currently comes with
>
>    [, awk, basename, cat, chmod, cmp, compress, cp, cut, diff, dirname,
>    egrep, expr, false, fgrep, find, gawk, grep, gzip, head, ln, ls,
>    mkdir, mv, pwd, reboot, rm, rmdir, sed, sleep, sort, tar, test,
>    touch, tr, true, uname, uniq, wc, and which.

Woow, excellent!  Do I get it right that “Gash Core Utils” refers to the
‘gash-core-utils’ branch at <https://gitlab.com/janneke/gash>?

So this is meant to be a Gash extension but separate from Gash.  Looks
really cool!

Do you plan to eventually merge it on Savannah?  (I expected to find the
code there.  :-)  BTW, I posted (unimportant) patches to
<https://lists.gnu.org/archive/html/gash-devel/2019-06/threads.html>.)

> that in comparison with Gash are much flakier.  While basic
> functionality is generally OK, most Gash tools cannot be used to
> bootstrap the entire system yet.  This holds especially for awk, grep,
> and sed.

(gash core-utils awk *) look quite fancy though!  I thought ‘configure’
only used a couple of trivial Awk snippets and after checking, I see
that there are in fact relatively fancy Awk programs in there.  Bah.

> Between autoconf 2.61 (2006) and autoconf-2.63 (2008), much
> functionality that was written in sed was moved to awk.  The NEWS file
> states performance reasons.  Gash's sed is much mature than Gash's awk,
> so currently it makes sense to target C versions of around 2004.
>
> However, some of the 2004 C tools are not good enough anymore to
> bootstrap the entire system; so we need a second round for them.  The
> Mes C Library is another constraint.  With Mes 0.21 it gained support
> for many early C tools; but more recent tools have C library
> requirements that are not yet covered.

How much work would be necessary at first sign to implement what’s
missing from (gash core-utils awk) to cover what modern-time ‘configure’
scripts need?

I suppose having a good(-enough) Awk implementation in Gash would be
more fruitful in the long run than reviving old C packages.  Notably, I
think you’d rather keep Mes’ libc as small as possible IMO, because it’s
cumbersome to write and maintain, which means more Scheme and less C.
But obviously, this all depends on the difficulty of implementing the
missing bits of Awk.

> Another thing to note is that we do not have bzip2, lzip or xz and that
> after 2009 some crucial tools (coreutils, diffutils, grep, sed, ...)
> start shipping .xz or .lz tarballs only.  While bzip2 can be built early
> in the bootstrap, I only managed to build xz with a fairly recent gcc
> (4.6).

At worst, we could host gzipped versions of these tarballs (or ask the
maintainers to do so).

Or we could have a fixed-output derivation that does the lzip->gzip
conversion (creating a “soft” circular dependency), which is kinda
equivalent to hosting a gzipped versions when substitutes are available.

Longer-term, we could also consider having a derivation built-in that
would allow us to “cheat” (i.e., take gz/lzip/xz/bzip2 for granted),
though it’s not so nice.

> Even with these considerations, there still is quite some room to change
> build order and versions of the C versions of coreutils&co.  The current
> choices are mostly made by "what works".  We could invest in fixing
> Gash's awk or sed or enrich the Mes C library or ..., if that seems a
> helpful thing to do.
>
> When we manage to merge this, we will have halved the bootstrap seed
> again, reducing the bootstrap seed to under 60MB.

Woohoo!

> Just when I thought the branch was functionally done; I already found one
> small problem; I have pushed this ugly workaround
>
> bootstrap: ACL: disable tests.  FIXME: LD_PRELOAD.
>
> Running
>
>     ./pre-inst-env build hello
>
> fails to build acl:
>
> ERROR: ld.so: object 
> '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from 
> LD_PRELOAD cannot be preloaded: ignored. != ~

We should have a debugging session for this one.  :-)

What about building the ‘core’ subset of ‘wip-bootstrap’ on berlin?
That would allow others to experiment (and debug!) without having to
build it all by themselves.

At some point we should consider “cleaning up” the history of that
branch.  For instance, I see commit “326d45561c gnu: Add gash.  WIP”,
which adds ‘guile-gash’ when there’s already a ‘gash’ package (and it
also modifies ‘jupyter-guile-kernel’).

We’ll also have to collectively review the new bootstrap tarballs.

Anyway, great perspectives and much excitement!  :-)

Thank you,
Ludo’.