Re: [libreplanet-discuss] observation re. build systems

[Thompson, David]

On Thu, Dec 31, 2015 at 1:28 PM, Miles Fidelman
<mfidelman@meetinghouse.net> wrote:
> Folks,
>
> I've recently found myself looking for a new distro for our production
> servers, and I've noticed something that seems just a bit disturbing:
>
> It seems like documentation of how to install and manage unpackaged software
> seems to have almost disappeared - i.e., it seems like awful lot of distros
> seem to assume that EVERYTHING is packaged.

It's the same on all UNIX-like systems for software that uses
something akin to the GNU build system:

     ./configure && make && make install

> At least in my experience, the reverse is more common:
>
> - developers tend to distribute source, built in their language-specific
> development environment, "packaged" for cross-platform building (e.g., a
> .tar file created using gnu autotools), or a .jar file, or what have you
>
> - it's pretty rare for developers to package for more than a few,
> particularly popular distros (if they package at all).
>
> - when building production servers, it's a lot more reliable to
> "./config; make; make install" than to rely on packages for anything other
> than utilities and platform stuff

I must disagree here.  That way lies madness.  For one thing, doing
this on each new node in a cluster slows down how quickly you can
provision new machine.  It also means that you cannot use the package
manager to facilitate upgrades or removals.  Sometimes it can't be
avoided for a variety of reasons (time constraints, usually), but I
try to create proper packages for as much custom software as possible.

> - an awful lot of stuff uses its own dependency resolution mechanisms
> and repositories (e.g., perl w/ cpan)

Yes, it's really terrible and causes lots of headaches for distro
maintainers.  I have personally been wrestling with RubyGems and NPM,
trying to figure out sane ways of packaging Ruby and Node.js software.

Unfortunately, the Ruby/Python/Node/Rust/Java/Go/etc. communities
think this is just fine, have no intention of changing, and cite
things like Windows compatibility as reasons to not bother with
GNU/Linux package managers.

I'm one of those DevOps people, so a lot of my professional time is
spent working on deployment automation, and by far the biggest problem
is that I need a special package manager for every single programming
language that is used.

> Somehow, this seems broken, and getting worse.  I've noticed this when it
> comes to systemd (our production systems rely on several applications that
> come with sysvinit scripts, but no other form of startup scripts.  Until
> now,
> we've been using Debian, and all works just fine - looks like I'll have
> to do a lot of tweaking for the next platform upgrade - which is what
> is motivating my looking around).  But, what with the emergence of several
> new platforms (e.g., SmartOS), it strikes me that the approach of
> distributing auto-configuring source, looks a lot less brittle than
> an assumption that someone is going to package everything, for every
> platform.

Software should never *require* a special manager to build and
install, but unfortunately the current reality is different thanks to
the recent wave of web developers that have trashed everything.

> Not sure what can be done, or by whom, but seems like an issue worth
> raising.
>
> Comments, thoughts?

Projects like GNU Guix are actively fighting against this craziness to
create a system that can be built entirely from source by anyone, in
an automated fashion, without having to trust random third-party
binaries that no one hows to reproduce.

On a more general note, take a look at the cross-distro Reproducible
Builds project.  Pushing for reproducibile, verifiable builds will
help the entire software industry up their game and resolve some of
the issues you are seeing as a side-effect of years of not caring.

https://reproducible-builds.org/

- Dave