Re: [PATCH] {maint} Document in detail some limitations of aclocal.

[Stefano Lattarini]

On Thursday 04 November 2010, Nick Bowler wrote:
> On 2010-11-04 22:28 +0100, Stefano Lattarini wrote:
> > On Thursday 04 November 2010, Nick Bowler wrote:
> > > On 2010-11-04 20:47 +0100, Stefano Lattarini wrote:
> > > > +AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [$1], [$2])])
> > > 
> > > This is insufficiently quoted, it should be:
> > >                         [m4_apply([AC_DEFUN], [[$1], [$2]])]
> > I'm not an m4 quoting guru, but I think that one single level of quotes
> > is sufficient to avoid *spurious* expansions.  Do you have any reason or
> > example showing that two levels of quotes are indeed needed?
> 
> "Insufficient quoting" was probably the wrong term to use here -- you
> are right that spurious expansion is avoided.   The problem is the use
> of m4_apply, which takes two arguments: the first is the (quoted) name
> of the macro, and the second is the (quoted) list of arguments.
Oh, I so misunderstood the m4_apply API!  You are perfectly right, and I've
amended the patch to address your observations.

The updated patch is attached; here is what I've squashed in:

-*-*-*-

diff --git a/doc/automake.texi b/doc/automake.texi
index cb65175..11fd76f 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -3756,7 +3756,7 @@ limitations with no clear workaround:
 $ @kbd{cat m4/foo.m4}
 # Indirection used here, to avoid triggering the bug described
 # above.
-AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [$1], [$2])])
+AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [[$1], [$2]])])
 $ @kbd{cat m4/bar.m4}
 MY_DEFUN([FOO], [BAR])
 $ @kbd{cat configure.ac}
diff --git a/tests/aclocal-limit-defun.test b/tests/aclocal-limit-defun.test
index 390f49d..4b4805a 100644
--- a/tests/aclocal-limit-defun.test
+++ b/tests/aclocal-limit-defun.test
@@ -47,7 +47,7 @@ grep -i 'uninitialized value.*\$1' stderr
 # But we can still work around the bug above, by "massaging" the input.
 
 cat > m4/foo.m4 <<'END'
-AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [$1], [$2])])
+AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [[$1], [$2]])])
 END
 
 # If we use use a literal `AC_DEFUN', aclocal should scan the definition
@@ -80,4 +80,15 @@ $AUTOCONF
 lastline=`sed -n '$p' configure`
 test x"$lastline" = x"FOO"
 
+# Finally, let's verify that everything would have worked out correctly
+# if aclocal had seen everything he could have been expected to see.
+
+cp m4/bar.m4 acinclude.m4
+
+$ACLOCAL -I m4
+$AUTOCONF
+
+lastline=`sed -n '$p' configure`
+test x"$lastline" = x"BAR"
+
 :
diff --git a/tests/aclocal-limit-include.test b/tests/aclocal-limit-include.test
index 7f2524c..46952a7 100644
--- a/tests/aclocal-limit-include.test
+++ b/tests/aclocal-limit-include.test
@@ -85,4 +85,15 @@ $AUTOCONF
 lastline=`sed -n '$p' configure`
 test x"$lastline" = x"FOO"
 
+# Finally, let's verify that everything would have worked out correctly
+# if aclocal had seen everything he could have been expected to see.
+
+cp m4/bar.m4 acinclude.m4
+
+$ACLOCAL -I m4
+$AUTOCONF
+
+lastline=`sed -n '$p' configure`
+test x"$lastline" = x"BAR"
+
 :

-*-*-*-

Thanks,
   Stefano

From 6d820c6f864e2725eaf53411d1168a3b59fa370f Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Thu, 4 Nov 2010 20:12:59 +0100
Subject: [PATCH] Document in detail some limitations of aclocal.

* doc/automake.texi (Limitations of aclocal): New section.
* tests/aclocal-limit-defun.test: New test, checking that the
aclocal limitation(s) exposed in the newly added manual section
does really hold.
* tests/aclocal-limit-include.test: Likewise.
* tests/Makefile.am (TESTS): Updated.
* .gitignore: Updated.
* THANKS: Updated.

From a report by Nick Bowler.
---
 ChangeLog                        |   13 +++++
 THANKS                           |    1 +
 doc/automake.texi                |   95 ++++++++++++++++++++++++++++++++++++++
 tests/.gitignore                 |    4 +-
 tests/Makefile.am                |    2 +
 tests/Makefile.in                |    2 +
 tests/aclocal-limit-defun.test   |   83 +++++++++++++++++++++++++++++++++
 tests/aclocal-limit-include.test |   88 +++++++++++++++++++++++++++++++++++
 8 files changed, 286 insertions(+), 2 deletions(-)
 create mode 100644 tests/aclocal-limit-defun.test
 create mode 100644 tests/aclocal-limit-include.test

diff --git a/ChangeLog b/ChangeLog
index 18c5ff6..2f4ba06 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2010-11-04  Stefano Lattarini  <address@hidden>
+
+       Document in detail some limitations of aclocal.
+       * doc/automake.texi (Limitations of aclocal): New section.
+       * tests/aclocal-limit-defun.test: New test, checking that the
+       aclocal limitation(s) exposed in the newly added manual section
+       does really hold.
+       * tests/aclocal-limit-include.test: Likewise.
+       * tests/Makefile.am (TESTS): Updated.
+       * .gitignore: Updated.
+       * THANKS: Updated.
+       From a report by Nick Bowler.
+
 2010-11-01  Ralf Wildenhues  <address@hidden>
 
        Add FAQ entry for bug reporting instructions.
diff --git a/THANKS b/THANKS
index 78da1de..1869d14 100644
--- a/THANKS
+++ b/THANKS
@@ -240,6 +240,7 @@ Motoyuki Kasahara   address@hidden
 Nathanael Nerode       address@hidden
 Nelson H. F. Beebe     address@hidden
 Nicholas Wourms                address@hidden
+Nick Bowler            address@hidden
 Nicolas Joly           address@hidden
 Nicolas Thiery         address@hidden
 NightStrike            address@hidden
diff --git a/doc/automake.texi b/doc/automake.texi
index b75c7a5..cb65175 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -188,6 +188,7 @@ Auto-generating aclocal.m4
 * Extending aclocal::           Writing your own aclocal macros
 * Local Macros::                Organizing local macros
 * Serials::                     Serial lines in Autoconf macros
+* Limitations of aclocal::      Some warts and limitations of aclocal
 * Future of aclocal::           aclocal's scheduled death
 
 Autoconf macros supplied with Automake
@@ -3099,6 +3100,7 @@ overridden using the @env{AUTOM4TE} environment variable.
 * Extending aclocal::           Writing your own aclocal macros
 * Local Macros::                Organizing local macros
 * Serials::                     Serial lines in Autoconf macros
+* Limitations of aclocal::      Some warts and limitations of aclocal
 * Future of aclocal::           aclocal's scheduled death
 @end menu
 
@@ -3702,6 +3704,99 @@ instance, if you have modified your local macros, do not 
expect
 system-wide versions.  If you want to do so, simply erase the local
 macros you want to revert, and run @samp{aclocal -I m4 --install}.
 
address@hidden Limitations of aclocal
address@hidden Limitations of @command{aclocal}
address@hidden Limitations of @command{aclocal}
address@hidden @command{aclocal}, Limitations
+
+Due to both intrinsic and historical reasons, @command{aclocal} is
+far from perfect.  The most noteworthy limitation, which macro authors
+and @command{aclocal} users should always be aware of, is that
address@hidden (in contrast to e.g.@: @command{automake}) doesn't
+work by consistently using m4 tracing, but must sometimes resort to
address@hidden the contents of the scanned files} when looking for macro
+definitions and file inclusions.  This means that perfectly valid m4
+constructs and usages might cause unexpected behaviours or even spurious
+failures.
+
+For example, the occurrence of the token address@hidden'', even if
+quoted and inside a macro definition, will confuse the grep-like scanner
+of @command{aclocal} into thinking that there is a macro definition to
+scan right away, and will cause an (internal!) @command{aclocal} error:
+
address@hidden Keep this example in sync with testcase 
"aclocal-limit-defun.test".
address@hidden
+$ @kbd{cat m4/foo.m4}
+AC_DEFUN([MY_DEFUN], [AC_DEFUN([FOO], [BAR])])
+$ @kbd{aclocal -I m4}
+Use of uninitialized value $1 in @dots{}
address@hidden example
+
+Similarly, the occurrence of an @code{m4_include} token, even if quoted
+and inside a macro definition, will confuse the grep-like scanner of
address@hidden into thinking that there is a file inclusion to
+process right away, and will probably cause an @command{aclocal} error:
+
address@hidden Keep this example in sync with testcase 
"aclocal-limit-include.test".
address@hidden
+$ @kbd{cat m4/foo.m4}
+AC_DEFUN([MY_INCLUDE], [m4_include([$1])])
+$ @kbd{aclocal -I m4}
+aclocal: m4/foo.m4:1: file `$1' does not exist
address@hidden example
+
+And while the bugs described above can be easily worked around
+by  extra indirections or ``creative quoting'', there still are
+limitations with no clear workaround:
+
address@hidden Keep this in sync with testcase "aclocal-limit-defun.test".
address@hidden
+## Let's see an @command{aclocal} limitation w.r.t.@: scanning
+## of macro definitons.
+$ @kbd{cat m4/foo.m4}
+# Indirection used here, to avoid triggering the bug described
+# above.
+AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [$1], [$2])])
+$ @kbd{cat m4/bar.m4}
+MY_DEFUN([FOO], [BAR])
+$ @kbd{cat configure.ac}
+AC_INIT([foo], [1.0])
+dnl: We would expect the following line to expand to `BAR' ...
+FOO
+$ @kbd{aclocal -I m4 && autoconf}
+# ... but it does not, since the @command{aclocal} scanner hasn't seen
+# the definition of @code{FOO} done through @code{MY_DEFUN}.
+$ @kbd{sed -n '$p'} configure
+FOO
address@hidden example
+
address@hidden Keep this in sync with testcase "aclocal-limit-include.test".
address@hidden
+## Now let's see an @command{aclocal} limitation w.r.t.@: scanning
+## of file inclusions.
+$ @kbd{cat m4/foo.m4}
+# ``Creative quoting'' used here, to avoid triggering the bug
+# described above.
+AC_DEFUN([MY_INCLUDE], [m4_][include([$1])])
+$ @kbd{cat m4/bar.m4}
+MY_INCLUDE([m4-extra/quux.m4])
+$ @kbd{cat m4-extra/quux.m4}
+AC_DEFUN([FOO], [BAR])
+$ @kbd{cat configure.ac}
+AC_INIT([foo], [1.0])
+dnl: We would expect the following line to expand to `BAR' ...
+FOO
+$ @kbd{aclocal -I m4 && autoconf}
+# ... but it does not, since the @command{aclocal} scanner hasn't seen the
+# inclusion of @file{m4-extra/quux.m4} through @code{MY_INCLUDE}, and thus
+# hasn't either seen the definition of @code{FOO} done in that file.
+$ @kbd{sed -n '$p'} configure
+FOO
address@hidden example
+
+These limitations might be lifted in a future version of
address@hidden, but then again, they might not, so you
+should be prepared to live with them.
 
 @node Future of aclocal
 @subsection The Future of @command{aclocal}
diff --git a/tests/.gitignore b/tests/.gitignore
index 3c1f990..5e97c89 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -1,5 +1,5 @@
-aclocal-*
-automake-*
+aclocal-1.*
+automake-1.*
 defs
 parallel-tests.am
 *.dir
diff --git a/tests/Makefile.am b/tests/Makefile.am
index acc7640..dc92176 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -59,6 +59,8 @@ acloca19.test \
 acloca20.test \
 acloca21.test \
 acloca22.test \
+aclocal-limit-defun.test \
+aclocal-limit-include.test \
 acoutnoq.test \
 acoutpt.test \
 acoutpt2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 117f2ba..042c3b2 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -326,6 +326,8 @@ acloca19.test \
 acloca20.test \
 acloca21.test \
 acloca22.test \
+aclocal-limit-defun.test \
+aclocal-limit-include.test \
 acoutnoq.test \
 acoutpt.test \
 acoutpt2.test \
diff --git a/tests/aclocal-limit-defun.test b/tests/aclocal-limit-defun.test
new file mode 100644
index 0000000..390f49d
--- /dev/null
+++ b/tests/aclocal-limit-defun.test
@@ -0,0 +1,83 @@
+#! /bin/sh
+# Copyright (C) 2010 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that an aclocal limitation w.r.t. macro defition, which is
+# described in the automake manual, does really hold.
+# Keep this test's content and name in sync with examples given
+# in the section "Limitations of aclocal" of the Automake manual.
+# If the aclocal limitation described there are lifted (either
+# completely or partially), updated this test accordingly.
+
+. ./defs || Exit 1
+
+set -e
+
+cat > configure.in <<'END'
+AC_INIT([foo], [1.0])
+FOO
+END
+
+mkdir m4
+
+# The use of an `AC_DEFUN' token, even quoted and in a macro definition,
+# confuse the grep scanner of aclocal into thinking that there really is
+# a macro definition to scan.
+
+cat > m4/foo.m4 <<'END'
+AC_DEFUN([MY_DEFUN], [AC_DEFUN([$1], [$2])])
+END
+
+$ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; }
+cat stderr >&2
+grep -i 'uninitialized value.*\$1' stderr
+
+# But we can still work around the bug above, by "massaging" the input.
+
+cat > m4/foo.m4 <<'END'
+AC_DEFUN([MY_DEFUN], [m4_apply([AC_DEFUN], [$1], [$2])])
+END
+
+# If we use use a literal `AC_DEFUN', aclocal should scan the definition
+# of `FOO' in bar.m4, and bring it in aclocal.m4.
+
+cat > m4/bar.m4 <<'END'
+AC_DEFUN([FOO], [BAR])
+END
+
+rm -rf autom4te*.cache aclocal.m4 configure # just to be sure
+
+$ACLOCAL -I m4
+$AUTOCONF
+
+lastline=`sed -n '$p' configure`
+test x"$lastline" = x"BAR"
+
+# If we use a macro that is *only* equivalent to `AC_DEFUN', aclocal
+# fails to scan it when looking for macro definitions.
+
+rm -rf autom4te*.cache aclocal.m4 configure # just to be sure
+
+cat > m4/bar.m4 <<'END'
+MY_DEFUN([FOO], [BAR])
+END
+
+$ACLOCAL -I m4
+$AUTOCONF
+
+lastline=`sed -n '$p' configure`
+test x"$lastline" = x"FOO"
+
+:
diff --git a/tests/aclocal-limit-include.test b/tests/aclocal-limit-include.test
new file mode 100644
index 0000000..7f2524c
--- /dev/null
+++ b/tests/aclocal-limit-include.test
@@ -0,0 +1,88 @@
+#! /bin/sh
+# Copyright (C) 2010 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that an aclocal limitation w.r.t. file inclusion, which is
+# described in the automake manual, does really hold.
+# Keep this test's content and name in sync with examples given
+# in the section "Limitations of aclocal" of the Automake manual.
+# If the aclocal limitation described there are lifted (either
+# completely or partially), updated this test accordingly.
+
+. ./defs || Exit 1
+
+set -e
+
+cat > configure.in <<'END'
+AC_INIT([foo], [1.0])
+FOO
+END
+
+mkdir m4 m4-extra
+
+# The use of an `m4_include' token, even quoted and in a macro definition,
+# confuse the grep scanner of aclocal into thinking that there really is
+# a file to include.
+
+cat > m4/foo.m4 <<'END'
+AC_DEFUN([MY_INCLUDE], [m4_include([$1])])
+END
+
+$ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; }
+cat stderr >&2
+grep 'foo.m4:1:.*\$1.*does not exist' stderr
+
+# But we can still work around the bug above, by "massaging" the input.
+
+cat > m4/foo.m4 <<'END'
+AC_DEFUN([MY_INCLUDE], [m4_][include([$1])])
+END
+
+cat > m4-extra/quux.m4 <<'END'
+AC_DEFUN([FOO], [BAR])
+END
+
+# If we use a literal `m4_include', aclocal should scan also the
+# contents of quux.m4 when looking for macro definitions.
+
+rm -rf autom4te*.cache aclocal.m4 configure # just to be sure
+
+cat > m4/bar.m4 <<'END'
+m4_include([m4-extra/quux.m4])
+END
+
+$ACLOCAL -I m4
+$AUTOCONF
+
+lastline=`sed -n '$p' configure`
+test x"$lastline" = x"BAR"
+
+# If we use a macro that is only equivalent to `m4_include', aclocal
+# fails to scan the contents of quux.m4 when looking for macro
+# definitions.
+
+rm -rf autom4te*.cache aclocal.m4 configure # just to be sure
+
+cat > m4/bar.m4 <<'END'
+MY_INCLUDE([m4-extra/quux.m4])
+END
+
+$ACLOCAL -I m4
+$AUTOCONF
+
+lastline=`sed -n '$p' configure`
+test x"$lastline" = x"FOO"
+
+:
-- 
1.7.1