[FYI] {master} aclocal: allow third-party macros take precedence over automake ones

[Stefano Lattarini]

It makes sense to allow "local system-wide" m4 macros take precedence
over built-in macros (defined in the aclocal versioned m4 directory
"${prefix}/share/aclocal-${APIVERSION}"), the same way we allow
"environment-level" m4 macros (through the ACLOCAL_PATH environment
variable ) take precedence over system-wide third-party macros.

An example: after this change, a definition of AM_PROG_VALAC placed
in file (say) '/usr/local/share/aclocal/my-vala.m4' should take
precedence over the same-named automake-provided macro defined in
file '/usr/local/share/aclocal-1.14/vala.m4'.

Even more importantly, after this change, m4 macros accessible
through $ACLOCAL_PATH entries will take precedence over m4 macros
defined in the aclocal versioned m4 directory; it was quite counter
intuitive that it was not the case before.

* aclocal.in (scan_m4_files): Search files in @system_includes before
files in @automake_includes.
* t/aclocal-acdir.sh: Adjust.
* t/aclocal-path-precedence.sh: Likewise.
* t/dirlist.sh: Likewise.
* doc/automake.texi: Adjust.
* NEWS: Update.

Signed-off-by: Stefano Lattarini <address@hidden>
---
 NEWS                         | 15 +++++++++++
 aclocal.in                   |  2 +-
 doc/automake.texi            | 64 +++++++++++++++++++-------------------------
 t/aclocal-acdir.sh           | 10 +++----
 t/aclocal-path-precedence.sh | 18 ++++++++-----
 t/dirlist.sh                 | 10 +++----
 6 files changed, 65 insertions(+), 54 deletions(-)

diff --git a/NEWS b/NEWS
index e672ad1..bc86fed 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,18 @@
+New in 1.14:
+
+* Aclocal search path:
+
+  - Third-party m4 files located in the system-wide aclocal directory,
+    as well as in any directory listed in the ACLOCAL_PATH environment
+    variable, now take precedence over "built-in" Automake macros.
+    For example, assuming Automake is installed in the '/usr/local'
+    hierarchy, a definition of the AM_PROG_VALAC macro found in file
+    (say) '/usr/local/share/aclocal/my-vala.m4' should take precedence
+    over the same-named automake-provided macro, as defined in file
+    '/usr/local/share/aclocal-1.14/vala.m4'.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 New in 1.13:
 
 * Version requirements:
diff --git a/aclocal.in b/aclocal.in
index 05b89e7..39351ad 100644
--- a/aclocal.in
+++ b/aclocal.in
@@ -411,8 +411,8 @@ sub scan_m4_files ()
       scan_m4_dirs (FT_USER, !$install, $user_includes[0]);
       scan_m4_dirs (FT_USER, 1, @user_includes[1..$#user_includes]);
     }
-  scan_m4_dirs (FT_AUTOMAKE, 1, @automake_includes);
   scan_m4_dirs (FT_SYSTEM,   1, @system_includes);
+  scan_m4_dirs (FT_AUTOMAKE, 1, @automake_includes);
 
   # Construct a new function that does the searching.  We use a
   # function (instead of just evaluating $search in the loop) so that
diff --git a/doc/automake.texi b/doc/automake.texi
index c0b1abf..3dcfcc5 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -3318,15 +3318,10 @@ way as it is for @command{automake} (@pxref{automake 
Invocation}).
 @cindex Macro search path
 @cindex @command{aclocal} search path
 
-By default, @command{aclocal} searches for @file{.m4} files in the following
-directories, in this order:
+By default, @command{aclocal} searches for @file{.m4} files in the
+following directories, in this order:
 
 @table @code
address@hidden @var{acdir-APIVERSION}
-This is where the @file{.m4} macros distributed with Automake itself
-are stored.  @var{APIVERSION} depends on the Automake release used;
-for example, for Automake 1.11.x, @var{APIVERSION} = @code{1.11}.
-
 @item @var{acdir}
 This directory is intended for third party @file{.m4} files, and is
 configured when @command{automake} itself is built.  This is
@@ -3334,14 +3329,19 @@ configured when @command{automake} itself is built.  
This is
 expands to @address@hidden@}/share/aclocal/}.  To find the compiled-in
 value of @var{acdir}, use the @option{--print-ac-dir} option
 (@pxref{aclocal Options}).
+
address@hidden @var{acdir-APIVERSION}
+This is where the @file{.m4} macros distributed with Automake itself
+are stored.  @var{APIVERSION} depends on the Automake release used;
+for example, for Automake 1.11.x, @var{APIVERSION} = @code{1.11}.
 @end table
 
 As an example, suppose that @command{automake-1.11.2} was configured with
 @address@hidden/usr/local}.  Then, the search path would be:
 
 @enumerate
address@hidden @file{/usr/local/share/aclocal-1.11.2/}
 @item @file{/usr/local/share/aclocal/}
address@hidden @file{/usr/local/share/aclocal-1.11.2/}
 @end enumerate
 
 The paths for the @var{acdir} and @var{acdir-APIVERSION} directories can
@@ -3363,8 +3363,8 @@ Any extra directories specified using @option{-I} options
 @enumerate
 @item @file{/foo}
 @item @file{/bar}
address@hidden @address@hidden
 @item @var{acdir}
address@hidden @address@hidden
 @end enumerate
 
 @subsubheading Modifying the Macro Search Path: @file{dirlist}
@@ -3391,13 +3391,14 @@ and that @command{aclocal} was called with the @samp{-I 
/foo -I /bar} options.
 Then, the search path would be
 
 @c @code looks better than @file here
address@hidden See test aclocal-dirlist.sh
 @enumerate
 @item @code{/foo}
 @item @code{/bar}
address@hidden @address@hidden
 @item @var{acdir}
 @item @code{/test1}
 @item @code{/test2}
address@hidden @address@hidden
 @end enumerate
 
 @noindent
@@ -3414,19 +3415,22 @@ If the @address@hidden option is used, then
 directories are
 
 @c @code looks better than @file here
address@hidden Keep in sync with aclocal-path-precedence.sh
 @enumerate
address@hidden @code{/usr/share/aclocal-1.11/}
 @item @code{/usr/share/aclocal/}
address@hidden @code{/usr/share/aclocal-1.11/}
 @end enumerate
 
-However, suppose further that many packages have been manually
-installed on the system, with $prefix=/usr/local, as is typical.  In
+However, suppose further that many packages have been manually installed
+on the system, with @address@hidden@}=/usr/local}, as is typical.  In
 that case, many of these ``extra'' @file{.m4} files are in
address@hidden/usr/local/share/aclocal}.  The only way to force
address@hidden/usr/bin/aclocal} to find these ``extra'' @file{.m4} files is to
-always call @samp{aclocal -I /usr/local/share/aclocal}.  This is
-inconvenient.  With @file{dirlist}, one may create a file
address@hidden/usr/share/aclocal/dirlist} containing only the single line
address@hidden/usr/local/share/aclocal}.  A way to force @file{/usr/bin/aclocal}
+to find these ``extra'' @file{.m4} files is to export @code{ACLOCAL_PATH}
+to @samp{/usr/local/share/aclocal}.  This is a little inconvenient,
+since it requires either explicit user cooperation, or editing of the
+system global shell initialization file.  With @file{dirlist}, one may
+create a file @file{/usr/share/aclocal/dirlist} containing only the
+single line
 
 @example
 /usr/local/share/aclocal
@@ -3435,20 +3439,18 @@ inconvenient.  With @file{dirlist}, one may create a 
file
 Now, the ``default'' search path on the affected system is
 
 @c @code looks better than @file here
address@hidden See test aclocal-dirlist.sh
 @enumerate
address@hidden @code{/usr/share/aclocal-1.11/}
 @item @code{/usr/share/aclocal/}
 @item @code{/usr/local/share/aclocal/}
address@hidden @code{/usr/share/aclocal-1.11/}
 @end enumerate
 
-without the need for @option{-I} options; @option{-I} options can be reserved
-for project-specific needs (@file{my-source-dir/m4/}), rather than
-using it to work around local system-dependent tool installation
-directories.
+without the need of any explicit @code{ACLOCAL_PATH} setting.
 
 Similarly, @file{dirlist} can be handy if you have installed a local
-copy of Automake in your account and want @command{aclocal} to look for
-macros installed at other places on the system.
+copy of Automake in your account and want @command{aclocal} to look
+for macros installed at other places on the system.
 
 @anchor{ACLOCAL_PATH}
 @subsubheading Modifying the Macro Search Path: @file{ACLOCAL_PATH}
@@ -3476,16 +3478,6 @@ Conversely to @file{dirlist}, @env{ACLOCAL_PATH} is 
useful if you are
 using a global copy of Automake and want @command{aclocal} to look for
 macros somewhere under your home directory.
 
address@hidden Planned future incompatibilities
-
-The order in which the directories in the macro search path are currently
-looked up is confusing and/or suboptimal in various aspects, and is
-probably going to be changed in the future Automake release.  In
-particular, directories in @env{ACLOCAL_PATH} and @address@hidden
-might end up taking precedence over @address@hidden, and
-directories in @address@hidden/dirlist} might end up taking precedence
-over @address@hidden  @emph{This is a possible future incompatibility!}
-
 @node Extending aclocal
 @subsection Writing your own aclocal macros
 
@@ -3513,7 +3505,7 @@ aclocal_DATA = mymacro.m4 myothermacro.m4
 @noindent
 Please do use @file{$(datadir)/aclocal}, and not something based on
 the result of @samp{aclocal --print-ac-dir} (@pxref{Hard-Coded Install
-Paths}, for arguments).  It might also be helpful to suggest to
+Paths}, for rationale).  It might also be helpful to suggest to
 the user to add the @file{$(datadir)/aclocal} directory to his
 @env{ACLOCAL_PATH} variable (@pxref{ACLOCAL_PATH}) so that
 @command{aclocal} will find the @file{.m4} files installed by your
diff --git a/t/aclocal-acdir.sh b/t/aclocal-acdir.sh
index 944604b..8b3f244 100755
--- a/t/aclocal-acdir.sh
+++ b/t/aclocal-acdir.sh
@@ -15,8 +15,8 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # Test aclocal's '--automake-acdir' and '--system-acdir' options.  Also
-# check that stuff in the automake acdir takes precedence over stuff in
-# the system acdir.
+# check that stuff in the system acdir takes precedence over stuff in
+# the automake acdir.
 
 . test-init.sh
 
@@ -65,14 +65,14 @@ $FGREP 'my--macro' configure
 
 rm -rf autom4te*.cache
 
-# Stuff in automake acdir takes precedence over stuff in system acdir.
+# Stuff in system acdir takes precedence over stuff in automake acdir.
 cat > am/bar.m4 <<'END'
 AC_DEFUN([MY_MACRO], [am--macro])
 END
 $ACLOCAL --automake-acdir am --system-acdir sys
 $AUTOCONF --force
 $FGREP 'fake--init--automake' configure
-$FGREP 'am--macro' configure
-$FGREP 'my--macro' configure && exit 1 # Just to be sure.
+$FGREP 'my--macro' configure
+$FGREP 'am--macro' configure && exit 1
 
 :
diff --git a/t/aclocal-path-precedence.sh b/t/aclocal-path-precedence.sh
index 0545231..9aee98a 100755
--- a/t/aclocal-path-precedence.sh
+++ b/t/aclocal-path-precedence.sh
@@ -46,10 +46,16 @@ cat > mdir3/bar.m4 << 'END'
 AC_DEFUN([BAR_MACRO], [::pass-bar::])
 END
 
-cat > mdir2/quux.m4 << 'END'
-AC_DEFUN([AM_INIT_AUTOMAKE], [::fail-init::])
-AC_DEFUN([AC_PROG_LIBTOOL],  [::pass-libtool::])
-AC_DEFUN([AM_GNU_GETTEXT],   [::pass-gettext::])
+cat > mdir2/quux-a.m4 << 'END'
+AC_DEFUN([AM_INIT_AUTOMAKE], [::pass-am-init::])
+END
+
+cat > mdir2/quux-b.m4 << 'END'
+AC_DEFUN([AC_PROG_LIBTOOL], [::pass-libtool::])
+END
+
+cat > mdir2/quux-c.m4 << 'END'
+AC_DEFUN([AM_GNU_GETTEXT], [::pass-gettext::])
 END
 
 cat > sysdir/libtool.m4 << 'END'
@@ -81,9 +87,9 @@ $FGREP '::pass-bar::' configure
 $FGREP '::pass-gettext::' configure
 $FGREP '::pass-libtool::' configure
 
-# Directories in ACLOCAL_PATH shouldn't take precedence over the internal
+# Directories in ACLOCAL_PATH shoul take precedence over the internal
 # automake acdir (typically '${prefix}/share/aclocal-${APIVERSION}').
-$FGREP 'am__api_version' configure
+$FGREP '::pass-am-init::' configure
 
 # A final sanity check.
 $FGREP '::fail' configure && exit 1
diff --git a/t/dirlist.sh b/t/dirlist.sh
index 2dc9038..6daf6a5 100755
--- a/t/dirlist.sh
+++ b/t/dirlist.sh
@@ -35,9 +35,9 @@ module=[$1]
 AC_SUBST(module)])
 END
 
-cat >dirlist-test/init.m4 <<EOF
-AC_DEFUN([AM_INIT_AUTOMAKE], [I should not be included])
-EOF
+cat >dirlist-test/init.m4 <<'END'
+AC_DEFUN([AM_INIT_AUTOMAKE], [Hey, I should be included, really!])
+END
 
 $ACLOCAL --system-acdir acdir
 $AUTOCONF
@@ -49,8 +49,6 @@ grep m4_include aclocal.m4 && exit 1
 
 grep 'GUILE-VERSION' configure
 
-# This bug can occur only when we do a VPATH build of Automake
-# but it's OK because VPATH builds are done by 'make distcheck'.
-grep 'I should not be included' configure && exit 1
+$FGREP 'Hey, I should be included, really!' configure
 
 :
-- 
1.8.0.1.409.g252f922