[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: outdated m4sugar

From: Akim Demaille
Subject: Re: outdated m4sugar
Date: Fri, 11 Jul 2008 10:38:18 +0200

Le 10 juil. 08 à 22:43, Eric Blake a écrit :

Hi Eric,

Right now, the latest bison sources are incompatible with the latest m4 sources. When I built bison with the latest m4.git (will become m4 1.6),


change from LIFO to FIFO m4wrap (done in order to make m4 obey POSIX)

I have not understood why they made that choice :(

trips up
a fresh build of bison:

$ m4 --version | head -n1
m4 (GNU M4) 1.5.89a.15-21859
$ sed -n '1,4 { s/<.*>/<...>/; p }' ChangeLog
2008-05-27  Joel E. Denny  <...>

        * NEWS (2.3b+): New section, empty for now.
        * configure.ac (AC_INIT): 2.3b -> 2.3b+.
$ make
make[2]: Entering directory `/home/eblake/bison/examples/calc++'
../../tests/bison -d -ra -S lalr1.cc -o ./calc++-parser.cc \
stdin:1: error: m4_init: unbalanced m4_divert_push:
stdin:1: m4_divert_push: KILL
stdin:1: the top level
/home/eblake/bison/src/bison: subsidiary program `/usr/local/bin/m4' failed
(exit status 1)

I tracked it to the fact that bison's m4sugar.m4 is three years old (it is newer than the m4sugar shipped with autoconf 2.59, but older than 2.59c), which means it relies on non-POSIX LIFO m4wrap semantics (autoconf 2.60 was the first non-beta release that works regardless of LIFO vs. FIFO m4wrap). ChangeLog-
wise, bison's version looks like it forked from autoconf commit
http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=commitdiff;h=cf3b3ede , then added the macro m4_prepend (which autoconf should also consider adding).

It would be very nice, indeed, that m4sugar be unified again.

In addition to Autoconf 2.62's m4sugar already working around the m4wrap issue, recent autoconf also has a number of speed optimizations and bug fixes in various m4sugar macros, which make autoconf execution noticeably faster than
what it was with autoconf 2.61 and earlier.

Bison is paying its performance penalty too to m4, so any speed
improvement will be warmly accepted!

 The latest autoconf m4sugar
provides a speed increase even when using GNU m4 1.4.5, independent from the speedups also possible by upgrading to m4 1.6. However, in improving m4sugar for autoconf 2.62, I also made some semantic changes on macros that were
previously undocumented.  I'm not sure whether blindly replacing
data/m4sugar/m4sugar.m4 with the latest version from autoconf, modulo the addition of m4_prepend and the change of one line regarding the sourcing of m4sugar/version.m4, will break anything in bison, since I'm not sure how many
of the undocumented m4sugar macros were in use by bison.

The test suite is reasonably strong, and I would be very surprised
if semantic incompatibilities would be unnoticed.

 But something needs
to be done to ensure that bison will continue to work after m4 1.6 is released, and to make it easier to keep m4sugar in sync between bison and autoconf (or
perhaps even move m4sugar into GNU m4 as the master).

Indeed, putting it in GNU m4 would be really great!  And actually,
autom4te was also meant to be part of m4.  I suppose that aclocal
is still part of Automake, and not of Autoconf, right?  Moving
things to the right place might not be so easy :)

 At any rate, I'm
debating about making bison a show-stopper for the release of m4 1.6 (ie. I'm reluctant to release m4 1.6 today, since it would mean the latest released
versions of bison and m4 would be incompatible).

Thank you for your attention to Bison!

 Autoconf has already proven
that m4sugar can be made to work across the m4 1.4.x/1.6 transition, so it would be nice to have a released version of bison that does likewise, allowing a user to upgrade to m4 1.6 without breaking bison (and so that if a user happens across a bison/m4 pair that fail to work together, the answer on either mailing list can be a simple "known issue; upgrade to the latest stable

Bison 2.4 is basically ready, betas were sent and AFAIK there
are no bug reports.  Maybe 2.4 could wait for the completed
conversion to a newer GNU M4?  Or would you advise to publish
2.4 as-is, and roll up a 2.5 for never M4s?

Also, I'm considering making m4 1.6 treat the -d command-line option in a
position-dependent manner.

Really?  I have been bitten by positional handling of the arguments
before.  What's the motivation?

 That means that bison's current use of 'm4
m4sugar/m4sugar.m4 - bison.m4 skeleton.m4 -dV" in output_skeleton
(src/output.c) will fail to turn on the desired tracing, since the - dV occurs too late in the command line. If you want full debugging for anything passed into the pipe to m4, the -dV should precede the "-" argument (although it can probably occur after the "m4sugar/m4sugar.m4" argument, since tracing the loading of m4sugar is not as interesting). Autoconf was immune to this issue,
since autom4te uses -d prior to any files.

OK, thanks for the prompt.

Meanwhile, bison uses a check for m4 1.3's ability to use frozen files
(m4/m4.m4, also borrowed from autoconf), even though bison itself does not appear to use frozen files. I don't know how well the current BSD m4 would fare with bison's m4 usage patterns, so I suspect that the test is bogus (feature-based tests to require GNU m4 over any other version should only be necessary if you actually rely on that feature). On the other hand, I know that the BSD folks have already taken pains to implement all GNU extensions that were in use by autoconf 2.59 except for frozen files, such that the BSD packaging of autoconf intentionally bypasses the m4/m4.m4 check to favor the BSD m4 by making a one-line tweak to autom4te to avoid the need for frozen files (at the expense of slower operation when using BSD m4 instead of GNU

The non-use of frozen files is probably just the result of laziness:
it's too hard to set up and use.  If we had autom4te instead of m4
directly, we would probably use frozen files.  I have no idea what
the speed-up would be though.

 I also know that while updating autoconf's version of m4sugar for
autoconf 2.62, I took some liberties to use GNU-specific constructs that require GNU m4 1.4.5 or newer, but which will fail with the current release of BSD m4. Thus, upgrading to a newer m4sugar may slightly hurt portability to BSD systems if the BSD m4 could previously be used to perform the processing
needed by bison's m4sugar, and where GNU m4 has not been installed.

That's not a problem, that's theirs.  We *require* GNU M4, and if
packagers want to do otherwise, that's none of our business.

 But you
can also argue that one of the features that will be introduced by GNU m4 1.6 (linear instead of quadratic execution speed on recursive macro expansion)

Wow :)  Where can we read about this?

means that introducing a hard dependency on GNU m4 is in the user's best interest, since BSD m4 already suffers from slower performance, and the performance gap between GNU and BSD only widens with the newer GNU m4 release.

I am willing to file copyright papers and help in the effort of updating bison's usage of m4sugar, if you would like, since I maintain GNU m4 and made
the bulk of the autoconf m4sugar improvements over the past 3 years.

Please, do!  You are most welcome!

reply via email to

[Prev in Thread] Current Thread [Next in Thread]