[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gnu-arch-users] Debian and package-framework
Re: [Gnu-arch-users] Debian and package-framework
Tue, 09 May 2006 16:50:21 -0700
Thunderbird 1.5 (X11/20060313)
So, as an accidental side effect of this discussion, I wound up writing
essay about package framework. I don't think it directly answers a
debian-maintainers concerns but I think the perspective might be
Why We Use `package-framework'
Sometimes people are surprised that we use `package-framework'
rather than `autoconf' to configure and build our programs.
Package-framework and the autoconf family of tools both make
it easier to write Makefiles and configuration scripts, but
they solve different problems.
The autoconf family is best for bootstrapping the first few
hard-to-port GNU programs in a new environment such as GNU
Make, GNU Bash, and GCC.
`package-framework' is better for most other programs:
programs that are not so hard to port and that can count on
the system to provide a Posix shell and GNU Make.
* autoconf Solves a Chicken and Egg Problem;
`package-framework' Makes a Chicken Omellete
Consider the problem of writing a configure script and
makefile for GNU Make and GNU Bash. There's a
chicken-and-egg problem. The makefiles can't assume that GNU
Make is already installed -- but the native versions of make
found on many systems differ from one another greatly.
Similarly, the configure scripts and the rules in makefiles
can't assume that GNU Bash is already installed, yet there
are awkward differences among the /bin/sh implementations
found on many systems. For those reasons, the autoconf tools
implement "compilers" for makefiles and shell scripts. These
read high level descriptions and emit truly portable shell
scripts and makefiles.
Those compilers are tricky to write and maintain. Worse,
the autoconf tools encourage users to extend these compilers
with new features, just for their own programs, and often
users do this incorrectly -- producing a configure script or
Makefile that will only work on a typical GNU/Linux system
but might not work on, say, a FreeBSD system. This is one of
the ways that when people use autoconf other than to bootsrap
those first few programs, the result can wind up being *less*
portable: exactly the opposite of the reason to use autoconf
in the first place!
Package-framework, on the other hand, requires a standard
Posix shell (Bash will do) and GNU Make. Those are pretty
small requirements. In exchange, package-framework works
without the need for shell script and makefile "compilers".
This makes package-framework smaller, simpler, and easier to
* `autoconf' Encourages Feature Tests,
`package-framework' Encourages Standard Code
Programs like GCC, Bash, and GNU Make were among the first
written and distributed by the GNU project. These programs
could not assume that the GNU C library would be available
because, at that time, it wouldn't even be complete for
several more years. Even today, no portable program is save
to assume that the GNU C library is available.
Back then many systems had C libraries and the usual header
files but there was a problem. Although POSIX standards
existed, few systems implemented those standards perfectly.
Some functions had different names on different systems.
Some header files were in different places. Some header
files and functions were missing entirely. Sometimes
everything was in the right place, but a function was buggy,
or had the wrong type. All of these problems still happen
today but to a much reduced extent: the POSIX standards are
more widely and more accurately implemented.
The variability between systems made it hard to write the
first core GNU programs in a way that would compile in many
different environments. A simple fix for one environment
could cause the build or the program to fail in another
environment. Generally speaking, people solve this kind of
problem four different ways:
1. conditional code based on system name
2. conditional code based on parameters from the user
3. conditional code based on feature tests -- probe the
system during configuration to see what's there
4. conservative coding and compatibility libraries
The GNU tools use a mix of all of those but autoconf places a
heavy emphasis on #3: feature tests. (GCC, of course, must
also use #1, especially learn about the CPU type.)
Feature tests are generally better than making guesses based
on the system name or requiring parameters from the user.
The resulting pre-processor conditionals tend to be simpler.
The result tends to be more robust and more easily ported.
#4 is almost always the best choice *when the option is
available*. For the early GNU programs, the option was often
not available. There was no one portable way to do proper
signal handling and subprocess management, for example, and
no way to really solve that by writing a compatibility
library. So autoconf placed a heavy emphasis on feature
tests and, to this day, encourages users to solve portability
problems that way.
`package-framework' is lucky to exist in a *slightly* simpler
world. It is much easier to write portable code, even for
many low-level system operations, without any pre-processor
conditionals at all. So, while `package-framework' does have
minimal support for feature tests, the emphasis is on
conservative coding and compatibility libraries.
* `autoconf' is Too Complicated
`package-framework' is Too Simple
The `autoconf' tools have been stretched far beyond their
original mission to help porting the first few GNU tools.
These days, they (it seems to us) are used for the vast
majority of C and C++ programs in the free software world.
`autoconf' has grown to be quite a bit larger than it
started. For some uses, it now depends on `perl'. It has
not gotten easier to maintain -- if anything, harder. Many
of us have experienced the frustration of `autoconf'-using
programs that won't configure or compile correctly other than
on a GNU/Linux system. Many of us have experienced the
frustration of having to have around multiple, incompatible
versions of `autoconf' because some programs require one
version while others require a different version.
`package-framework', more or less, does one simple thing
well. It is already very simple and, even so: there is more
code than needs to be there for the current functionality.
If it changes in the future, is should become even simpler.
But `package-framework' is in some ways a little bit *too*
simple. Right now, it offers no help at all building or
linking against dynamically loaded libraries -- a major
deficiency. Additionally, it has no support for finding a
required version of an installed library in the case where
there may be more than one version installed. These
weaknesses are why, for example, a particular version of
`libneon' is bundled with GNU Arch and why that library is
statically linked to Arch.
* A Possible Future
`package-framework' generally serves us well and we think
that its basic design is sound. We believe that `autoconf'
is over-stretched as it is popularly used and that it
encourages people to use feature probes where more
conservative coding practices would be a better solution. We
emphatically recognize that `autoconf' still does, and may
"forever" play an important role in bootstrapping the most
basic GNU system tools.
The `package-framework' implementation today is a little bit
messy. It would be an improvement to carefully review the
scripts and remove "dead code" and some minor "features" that
serve no useful purpose. It would be an improvement to
carefully review the Makefile library and make similar
changes. The result would be a smaller, sleaker
implementation of the current functionality.
Next: `package-framework' needs support for dependencies on
versioned libraries and for dynamic library loading.
Versioned library support is a small matter of writing some
new shell code -- there is no great mystery to it. Dynamic
loading is tricky because, indeed, this is an area where
portability is hard to achieve. It is not clear that
adopting `libtool' directly is the right answer: `libtool'
needs to be studied from a `package-framework' perspective
and either adopted or reverse engineered and replaced.
Next: There is a need for improved standards for packaging
source distributions of software packages. On all GNU
systems, there should be a standard way to determine where
to install programs and to tell what other programs have
already been installed. Configure/build tools are the
right place to implement such standards.
Finally, it is difficult to imagine an easy way to get many
projects to switch to `package-framework' or to adopt new
standards for source packages. People have an awful lot
invested in the tools they already use. Vendors display only
weak interest in that kind of standard. We have many legacy
packages, already very hard to configure and build, for which
the effort of switching over would be monumental.
Nevertheless, these are problems of the sort that can be
meaningfully and usefully addressed in "one small step at a
time" mode. What will give people incentive to make the
switch? To have very good tools that make the switch easier
to stomach, and to be offering very good benefits that result
from switching. Step 1: work on the tools, keeping the need
for benefits in mind.
Sylvain Beucler wrote:
On Mon, May 08, 2006 at 08:01:40PM +0200, Sylvain Beucler wrote:
I have a few questions with regard to packaging / package-framework,
could you help us?
I think I can answer some of them now, feel free to correct me :)
- documentation: read comments in
- cross-compilation: this would require passing --host/--build
../configure options down to PLUGIN/AUTOCONF-based components, and
introducing a caching or best-guess mecanism in hackerlab's
compile-time tests afaics - so I can forget about it.
- using externals libs instead of PLUGIN ones (for libneon): that one
I still wonder about :) Is it supported by package-framework somehow,
or will I have to trick the system? (eg. via some hand-made Makefile
rules that would set gcc -I and ld -L appropriately?)
Re: [Gnu-arch-users] Debian and package-framework, Thomas Lord, 2006/05/09
- [Gnu-arch-users] Debian and package-framework, Sylvain Beucler, 2006/05/08
- Re: [Gnu-arch-users] Debian and package-framework, Andy Tai, 2006/05/08
- Re: [Gnu-arch-users] Debian and package-framework, Sylvain Beucler, 2006/05/09
- Re: [Gnu-arch-users] Debian and package-framework,
Thomas Lord <=
- [Gnu-arch-users] Re: Debian and package-framework, Miles Bader, 2006/05/09
- Re: [Gnu-arch-users] Debian and package-framework, Ludovic Courtès, 2006/05/10
- Re: [Gnu-arch-users] Debian and package-framework, Thomas Lord, 2006/05/10
- [Gnu-arch-users] Re: Debian and package-framework, Miles Bader, 2006/05/10
- Re: [Gnu-arch-users] Re: Debian and package-framework, Stephen J. Turnbull, 2006/05/11
- Re: [Gnu-arch-users] Re: Debian and package-framework, Stefan Monnier, 2006/05/11