[bug#57050] [PATCH v2 05/13] gnu: racket: Update to 8.6.

[Liliana Marie Prikler]

Am Donnerstag, dem 11.08.2022 um 18:40 -0400 schrieb Philip McGrath:
> Hi,
> 
> On Thu, Aug 11, 2022, at 7:44 AM, Liliana Marie Prikler wrote:
> > Am Donnerstag, dem 11.08.2022 um 07:08 -0400 schrieb Philip
> > McGrath:
> > 
> > > Also, update 'chez-scheme-for-racket' to 9.5.9.2.
> > As with Zuo, can this be split into a separate patch?  If not, why
> > does it get mentioned here as something noteworthy rather than just
> > being part of the ChangeLog?
> > 
> 
> This absolutely should not be split into more patches. While I split
> out Zuo as you requested, I continue to believe that it was better as
> a single patch.
Belief is one thing, but I think you're cherry-picking commits to not
have said beliefs questioned, which imho is a dangerous thing to do.

> More broadly, I hope to put together a 'racket-build-system' before
> the 8.7 release. Whenever that happens, it will mean turning the
> inputs of the 'racket' and 'racket-minimal' packages into, last I
> counted, 203 additional packages.
That's great.  I hope this will also let us unvendor some of racket
itself.

> I think it would be bad for everyone—patch writers, patch reviewers,
> users, and people reading the history later—to break Racket releases
> into a deluge of little patches
Well, first of all that's debatable, second we do have wip branches for
big workloads like gnome, r, and some others, and I think racket would
probably fit into that category.  Thus, users sitting on the main
branch would only observe the "merge wip-racket into master" commit,
whereas reviewers can take their time to observe each package in
isolation.

> combinations of technical and social mechanisms to make certain
> potentially problematic states difficult to fall into.
Just call it a monorepo.

> A Racket release is an update to a family of packages that are
> developed and released together. Like a TeX Live update (cf.
> ee25e3fcab9d2e24c2826b771b52d797c152193b) or the KDE Frameworks
> libraries (cf. 9d3965edca29f80667374da45214cc6f22a85be4), the
> contents of a Racket release should be updated together. I wouldn't
> insist on one commit in all imaginable circumstances, but I think it
> should be the norm. That's all the more true of Racket components
> that share a canonical Git repository, for which Racket tools take
> steps to warn you if not fail if you have mismatched versions.
The difference between the two commits you mention and the one you
provide is that the upgrades to those derived packages mostly come for
free, whereas yours doesn't.  Compare 

> * gnu/packages/kde-frameworks.scm (extra-cmake-modules, attica,
> bluez-qt,
> breeze-icons, kapidox, karchive, kcalendarcore, kcodecs, kconfig,
> kcoreaddons,
> kdbusaddons, kdnssd, kguiaddons, kholidays, ki18n, kidletime,
> kirigami,
> kitemmodels, kitemviews, kplotting, ksyntaxhighlighting,
> kwidgetsaddons,
> kwindowsystem, modemmanager-qt, networkmanager-qt, oxygen-icons,
> prison,
> qqc2-desktop-style, solid, sonnet, threadweaver, kactivities, kauth,
> kcompletion, kcontacts, kcrash, kdoctools, kfilemetadata,
> kimageformats,
> kjobwidgets, knotifications, kpackage, kpty, kunitconversion,
> syndication,
> baloo, kactivities-stats, kbookmarks, kcmutils, kconfigwidgets,
> kdeclarative,
> kded, kdesignerplugin, kdesu, kdewebkit, kemoticons, kglobalaccel,
> kiconthemes, kinit, knewstuff, knotifyconfig ,kparts, kpeople,
> krunner,
> kservice, ktextwidgets, kwallet, kxmlgui, kxmlrpcclient, purpose,
> kde-frameworkintegration, kdelibs4support, khtml, kjs, kjsembed,
> kmediaplayer,
> kross): Update to 5.70.0.
to
> (chez-scheme-for-racket): Update to 9.5.9.2.
> [native-inputs]: Add 'zuo'.
> [arguments]<#:out-of-source?>: Use out-of-source build.
> <#:tests?>: Skip them due to ongoing problems.
> <#:configure-flags>: Add '--install-csug=' and '--
> installreleasenotes='.
> <#:make-flags>: Use 'zuo' from 'native-inputs'. Supply 'STEXLIB='
> here,
> rather than in a phase.
> <#:phases>: Replace 'install-docs' using new 'make' target.

> I also want to emphasize that splitting up patches is not free.
I also want to emphasize that reviewing patches is not free.

> Splitting/rebasing v2 of this series took hours of work. Even
> specifically in splitting off Zuo, I made a (trivial) error in my
> first version of this particular patch and had to amend the commit
> and rebuild. Splitting this any further would get farther away both
> from how I actually wrote and tested this patch series over the past
> four months and from the way Racket is developed and released. It
> would take a non-negligible amount of effort, all to produce a result
> that I believe would be worse.
As someone who's authored larger series myself, I can somewhat
understand your pain.  However, I don't think the development
guidelines of any particular package ought to influence how we do
things in Guix.  Otherwise, we'd have adopted emoji commit messages for
some of them by now.

> I mention the version number for 'chez-scheme-for-racket' because it
> shows up relatively prominently in Guix tooling and even e.g. in the
> path of the package documentation. I don't know why it would be a
> problem to do so, but I would vastly prefer to remove the mention
> than to split the patch, if it really has to be a choice. The Racket
> release notes don't mention the corresponding version of Chez Scheme.
If chez and racket were that tightly coupled, you'd think they go
together without mention, no?  For instance, when upgrading Emacs to
28.1, I don't have to mention all the built-in packages that've been
updated, you get those from the release notes.

However, looking at the ChangeLog above, I don't think these two are
tightly enough coupled to just say that simultaneous upgrades come for
free (as they do with Emacs, but also Renpy, WebkitGTK, and a bunch of
others that I'm not personally involved with).

> > > [...]
> > > @@ -448,18 +456,52 @@ (define-public chez-scheme-for-racket
> > >         (delete "libx11" "util-linux:lib")))
> > >      (native-inputs
> > >       (modify-inputs (package-native-inputs chez-scheme)
> > > +       (append zuo)
> > >         (replace "chez-scheme-bootstrap-bootfiles"
> > >           chez-scheme-for-racket-bootstrap-bootfiles)))
> > Prefer prepend over append.
> > 
> 
> Why?
Prepend is a cons internally, so it's both faster and more natural. 
Alphabetic ordering doesn't matter here and I hope neither do search
paths in your use case.

> > >      (arguments
> > >       (substitute-keyword-arguments (package-arguments chez-
> > > scheme)
> > > +       ((#:out-of-source? _ #f)
> > > +        #t)
> > > +       ((#:tests? _ #t)
> > > +        ;; FIXME: There have been some flaky test failures. Some
> > > have
> > > been
> > > +        ;; fixed upstream post-release but have proven non-
> > > trivial to
> > > +        ;; backport; at least one issue remains. Re-enable tests
> > > once
> > > +        ;; https://github.com/racket/racket/issues/4359 is
> > > fixed.
> > > +        #f)
> > Rather than skipping tests altogether, skip just the flaky ones.
> > 
> 
> If I knew how to do that, I absolutely would. Well, short of making
> the `process` function silently fail to run `/bin/sh` again, maybe.
> If you look at the linked issue, you'll see that I've been chasing
> down failures for the last month. For a while I had cherry-picked
> 153ff9acb7ad63717a50bb26cd5aaa053870c666, which fixed the only issue
> with the implementation (a race condition in one mode of the garbage
> collector), but it didn't seem worth carrying the patch when things
> were still failing. The other issues seem to be problems with running
> the test suite. Probably they appeared now because we hadn't actually
> been running as much of it as we seemed to be. So far, I haven't had
> failures building with current master, but there aren't any commits
> touching anything obviously related, and Unicode 14 and grapheme
> cluster support have landed since the release branch, so it wouldn't
> be reasonable to just sweep in all the changes indiscriminately.
> Further, it's at least possible that I may just have been winning the
> race recently and that the actual problem might still be there. The
> Chez test suite takes an hour to run (maybe more), and I haven't been
> able to reproduce the failures outside of Guix, so it's not exactly
> rapid iteration. All in all, I don't know any change to make other
> than turning off tests that I can feel confident will work reliably
> for Guix users.
There surely must be more than a single test file, no?  The typical way
to do this in autotools would be to specify TESTS="the tests to run",
or to (substitute* "tests/Makefile.am" ((" bad-test") "")).  HTH

> For what it's worth, difficulty running the Chez tests is not unique
> to Guix. In the course of adding Zuo, Matthew Flatt commented, "After
> this conversion, I was able to run the Chez Scheme test suite on
> Windows for the first time; it must have been possible before, but I
> never quite got the right tools installed in the right way to make it
> work." [1] AFAICT Debian skips the Chez test suite unless it is
> specifically requested [2]: I don't know why, maybe just because it
> takes so long.
> 
> [1]:
> https://github.com/racket/racket/pull/4179#issuecomment-1094137092
> [2]:
> https://salsa.debian.org/scheme-team/chezscheme/-/blob/fbfa9c35db184f5b22ac6c0058d754e1a45f5f68/debian/rules#L66-69
That's horrible.

> > > @@ -130,12 +132,12 @@ (define-module (gnu packages racket)
> > >  ;; output. The function 'racket-vm-for-system' returns the
> > > recomended Racket
> > >  ;; VM package for a given system.
> > >  ;;
> > > -;; The file 'racket.scm' builds on these packages to define
> > > 'racket-
> > > minimal'
> > > -;; and 'racket' packages. These use Racket's support for
> > > ``layered
> > > -;; installations'', which allow an immutable base layer to be
> > > extended with
> > > -;; additional packages. They use the layer configuration
> > > directly
> > > provide
> > > -;; ready-to-install FHS-like trees, rather than relying on the
> > > built
> > > in
> > > -;; ``Unix-style install'' mechanism.
> > > +;; We then define the packages 'racket-minimal' and
> > > +;; 'racket'. These use Racket's support for ``layered
> > > installations'', which
> > > +;; allow an immutable base layer to be extended with additional
> > > packages.
> > > +;; They use the layer configuration directly provide ready-to-
> > > install FHS-like
> > > +;; trees, rather than relying on the built in ``Unix-style
> > > install''
> > > +;; mechanism.
> > >  ;;
> > >  ;; Bootstrapping Racket:
> > >  ;; ---------------------
> > This is a leftover from 8.6, but do the "FHS-like" installations
> > actually adhere to the FHS or just to the bit that says "opt means
> > we
> > can do whatever we want 😎️"?
> > 
> 
> The 'racket' and 'racket-minimal' packages follow FHS. (Pedantically,
> because compiled Racket code may or may not be architecture specific,
> and used to always be architecture-independent, some configuration
> modes put things in "share" that belong in "lib", but the current
> upstream "Unix-style" defaults and out packages do not do that.)
> 
> Here's a comparison of the hierarchy of our non-VM Racket packages
> vs. an in-place Racket installation, with the bracketed numbers
> indicating how the two match up:
> 
> /gnu/store/xyz-racket-8.5/
> ├── bin/ [1]
> ├── etc/
> │   └── racket/
> │       └── config.rktd [2]
> ├── lib/
> │   └── racket/ [3]
> │       ├── links.rktd
> │       ├── mans.rktd
> │       └── pkgs/ [4]
> └── share/
>     ├── applications/ [5]
>     │       └── drracket.desktop
>     ├── doc/
>     │       └── racket/ [6]
>     │           └── index.html
>     ├── man/ [7]
>     │       └── man1/
>     │           └── racket.1.gz
>     └── racket/ [8]
>              └── infocache.rktd
> 
> /gnu/store/xyz-racket-vm-cs-8.5/opt/racket-vm/
> ├── bin/ [1]
> ├── collects/ [would be share/racket/collects, but not duplicated in
> layers]
> ├── doc/ [6]
> ├── etc/
> │   └── config.rktd [2]
> ├── include/
> │   ├── chezscheme.h
> │   ├── racketcsboot.h
> │   └── racketcs.h
> ├── lib/ [3]
> │   ├── libracketcs.a
> │   ├── petite.boot
> │   ├── racket.boot
> │   └── scheme.boot
> ├── man/ [7]
> │   └── man1/
> └── share/ [8]
>     ├── applications/ [5]
>     └── pkgs/ [4]
Your [8] seems broken.

But understanding this correctly, we'd have to move vm collects to lib
or share and man+doc to share?  This doesn't look too bad, but I bet
there's some complication with layers.
> 
> > >  ;;
> > > -;; Code:
> > > +;; Zuo is notably *not* a problem for bootstrapping. The
> > > implementation is a
> > > +;; single hand-written C file designed to build with just `cc -o
> > > zuo
> > > zuo.c`,
> > > +;; even with very old or limited compilers. (We use the Autoconf
> > > support for
> > > +;; convienience.)
> > > +;;
> > > +;; CODE:
> > Is that something that needs be mentioned?
> > 
> 
> Some people have asked whether the Zuo implementation does or should
> share things with Racket and/or Chez Scheme. It seemed worth noting
> that it intentionally does not, for the sake of bootstrapping.
In reply to this bug?  In Chez/Racket spaces?  I think the package
description for zuo should make it clear that it can be used for
bootstrapping, so from a Guix side that comment seems superfluous.

Cheers