Re: packaging lilypond as a docker container?

[Jonas Hahnfeld]

Let me respond to some of your points:

Am Mittwoch, den 22.01.2020, 11:00 +0100 schrieb Han-Wen Nienhuys:
> Let me explain my thinking more in detail. (I've CC'd Jan in case he wants to 
> give further historical insights) 
> 
> 
> The objective of GUB was to provide installers for LilyPond with the 
> following requirements:
> 
> 1) build for all platforms (OSX, Windows, Linux) with minimal effort, without 
> having to manage windows and OSX machines.
> 
> 2) all platforms use exactly the same versions of all software, so everyone 
> has the same experience, and problems always reproduce across platforms.
> 
> 1) implied setting up a cross compiling environment. This is a pain in the 
> ass, because GCC must be compiled for each (host, target) pair separately. 
> This is very slow and takes up a lot of space.

And doesn't even work. Did you recently try to run the binaries
produced for FreeBSD? They're dynamically linked to GUB's glibc which
isn't packaged up. Not surprisingly, FreeBSD has its own libc with
different so-versions and so on.

> However, the major reason why this is a PITA is that many pieces of software 
> don't support cross-compiling well. In particular, some software runs itself 
> as part of the build (lilypond is an example, but so do Python and Fontconfig 
> IIRC). Making them cross-compile is something that Jan did in his 
> professional career (embedded software!), so he knew how to do it for 
> LilyPond. It is however very intricate, and time consuming to test. One has 
> to pore over compilation logs, understand what all these packages are trying 
> to do, and then come up with ways to do the same in a cross-compiling manner. 
> This is why the GUB specifications are patching compilation steps in creative 
> ways.

The patches are horrible to maintain once you want to bump more than
just to a new patch version.

> I found it demoralizing to work on, because I want to work on music 
> typesetting, and not on patching up someone else's wonky compilation script.
> 
> Because we now have a very expensive and complicated setup (GCC !), it makes 
> sense to utilize this setup across builds. For example, if we upgrade the 
> lilypond version, we'd rather not have to rebuild GCC from scratch. So, GUB 
> has a packaging system, where you can upgrade LilyPond (or any intermediate 
> dependency) without starting from scratch.

I do the same for the dependencies of LilyPond, except for all that
complicated packaging stuff. If you screw something up, just rebuild
it. This takes around ~20 minutes on my Laptop from 2016.

> We started out with a set of shell scripts, but they were hard to maintain, 
> so we built GUB. So, to me, Jonas' proposal to have a set of simple 
> compilation scripts is a little funny.

As a predecessor of GUB, this probably involved cross-compilation? I'm
not doing this, it's just native compilation all way long.

> Cross compiling has these problems above, but the "native" compile also has 
> problems. One has ensure that the native compile doesn't pick up dependencies 
> from the system it is compiled from, as that could cause unpredictable 
> behavior. You'd run GUB on a machine that is used for LilyPond, so all of the 
> dependencies can be consumed from both GUB and the host machine, and it is 
> easy for something to go wrong here.

I think this got much better, and I looks like I finally convinced pkg-
config not to use anything from the system. This only leaves the
compiler headers and libc. All of the rest is built as static libraries
which end up in the LilyPond executable.

> In the intervening 15 years, many things have changed. 
> 
> A negative: apple licensing restrictions say that we can't cross-compile to 
> OSX (although to be honest, I never read the terms & conditions for the XCode 
> stuff I downloaded from apple back in the day. Maybe we've always been in 
> violation of some T&C)
> 
> A posiitive: all major platforms run on x86 64-bit CPUs, and there are Open 
> Source VM solutions for OSX and Windows. So, a VM running Linux on OSX or 
> Windows can be almost as fast as running Linux directly. This makes GUB's 
> cross compilation to other architectures superfluous.
> 
> A positive: Linux now supports namespaces, and in particular: file system 
> namespaces. This makes it possible to build a native Linux package fully 
> isolated from its host system, thus skipping GUB's laborious GCC setup. This 
> also lets you run a regular package manager on the namespaced file system, 
> and this is essentially what Docker gives you: a way to install arbitrary set 
> of dependencies and use them. This makes GUB's package manager superfluous.
> 
> A positive: Docker is open source, and extremely popular technology. Docker 
> does the hard work of packaging up VM + Docker for windows and OSX as 
> installer, and providing a management/distribution mechanism for images.
> 
> All in all, I think we could use Docker for distributing LilyPond. Then we 
> would have a distribution mechanism that works for OSX  and Windows, and it 
> has no overhead to compiling LilyPond as normal Linux package. Because we can 
> control the base image, we can precisely control the dependencies, and be 
> sure that the binary behaves the same across all OSes. The best is that 
> building releases won't require any arcane knowledge of cross-compilation 
> quirks.

So you want to require users to setup Docker on Windows to run
LilyPond? Not the lowest barrier we could possibly have IMO.

Jonas

> 
> GUIX is Jan's current project. I think it has some similarities to GUB, but 
> it is focused on providing an environment where all binaries are reproducibly 
> built from source. This is much more ambitious than GUB, and seems overkill 
> compared to what we need for LilyPond. I think using it also entails many 
> more compilation steps, which would makes the release process slow again.
> 
> 
> 
> On Tue, Jan 21, 2020 at 11:28 AM David Kastrup <address@hidden> wrote:
> > Jonas Hahnfeld <address@hidden> writes:
> > 
> > > Am Dienstag, den 21.01.2020, 02:38 -0600 schrieb Karlin High:
> > >> On 1/21/2020 1:49 AM, Han-Wen Nienhuys wrote:
> > >> > if GUB is used, who is maintaining and/or working on it?
> > >> 
> > >> Almost exactly a year ago, there was a sizable "Please test gub" effort 
> > >> initiated by Knut Petersen.
> > >> 
> > >> <
> > >> https://lists.gnu.org/archive/html/lilypond-devel/2019-01/msg00221.html
> > >> >
> > >> 
> > >> Clear instructions were given for exactly how to test GUB, and just that 
> > >> thread seems to have involved about 16 people.
> > >> 
> > >> More recently, it looks like Jonas Hahnfeld's work with Python 3 
> > >> migration involved quite a number of changes to GUB.
> > >
> > > Yep, and it's not been a very pleasant experience. To be precise here:
> > > It's not about porting GUB to Python 3, it's just that porting LilyPond
> > > will introduce a dependency on an updated package for the next release.
> > > So while David is mostly correct that contributors don't need to bother
> > > with GUB, that doesn't hold true once you want to bump one of its
> > > dependencies...
> > >
> > > Based on this experience, I've thought about how to improve the process
> > > of building binary releases for LilyPond. What I have been
> > > experimenting with is a set of portable shell scripts that build mostly
> > > static executables. I've a prototype ready at 
> > > https://github.com/hahnjo/lilypond-binaries which works for Linux and
> > > FreeBSD. I don't see a reason it cannot work on macOS, but I don't have
> > > the possibility to test...
> > 
> > The problem is "on macOS" since current macOSX library/toolbox licenses
> > all prohibit use in cross-building environments.  It looks like a given
> > that we will not be able to offer macOSX libraries in future as the
> > result of a unified release process.  Our sources build reasonably well
> > that macOSX integrators using some of the macOSX typical packaging
> > systems are able to provide reasonable replacements.
> > 
> > But the elephant in the room is Windows.  Han-Wen says he never wants to
> > touch GUB again (and he actually didn't as far as I remember) but I
> > don't want to touch Windows again, and GUB provides them.  The
> > dependency nightmare on a non-UNIX like platform was what brought GUB
> > into being in the first place: native builds are much more of an ongoing
> > problem.  Just look at the continuous effort the Git project has in
> > keeping a Windows version available.
> > 
> > >>  From the notes I've kept, Jan Nieuwenhuizen proposed replacing GUB with 
> > >> something based on Guix.
> > >> 
> > >> <
> > >> https://lists.gnu.org/archive/html/lilypond-user/2016-09/msg00690.html
> > >> >
> > >> 
> > >> That idea was mentioned a few times since, but I can't recall if 
> > >> GUB-replacement discussions have moved beyond that or not.
> > >
> > > As far as I understand, Guix is a package manager in the first place.
> > > In the thread Jan proposes to use it for cross-compilation, which I
> > > think is the primary reason for the complexity in GUB. So why go that
> > > route once more?
> > > Instead I designed the mentioned scripts such that they produce native
> > > executables. That's also how TeXlive builds their packages, for
> > > example, and it works quite well if you compile on the oldest OS that
> > > you intend to support (CentOS 7, FreeBSD 11).
> > >
> > > I haven't decided yet how to compile for Windows. Maybe that's still a
> > > valid use case for cross-compilation (but only with a very limited
> > > scope).
> > 
> > Windows really is the elephant in the room.  MacOSX will cater with
> > native port systems like MacPorts etc and other UNIX-like systems also
> > have working packagers and package systems.
> > 
> > > If this attempt sounds interesting to the community, I would be happy
> > > to submit the scripts for inclusion into the LilyPond repository
> > > itself. That would also solve another issue with GUB: Currently it's a
> > > separate repository with no way to keep it in sync with changing
> > > dependencies for LilyPond...
> > 
> > Maybe we should entertain two branches of GUB matching current stable
> > and unstable release tracks?  Or otherwise deal with a difference in
> > dependencies for stable/unstable?
> > 
> > -- 
> > David Kastrup
> 
> 
>

signature.asc
Description: This is a digitally signed message part