bug#19370: Acknowledgement (LT 2.4.4 regression (vs. 2.4.2))

[Pavel Raiskup]

Hi Gary, quite long, sorry for that,

On Monday 22 of December 2014 21:22:01 Gary V. Vaughan wrote:
> tags 19370 notabug
> close 19730
>
> On Dec 20, 2014, at 11:18 AM, Jeff Squyres (jsquyres) <address@hidden> wrote:
> > Hopefully that's enough to get you going in the right direction.
> 
> Indeed it is.  And the problem is that autoreconf, as called from the
> autogen.sh in the tarball, still runs the tools in the wrong order.
> Autoreconf stupidly runs aclocal first, and then calls libtoolize which
> adds more m4 files to AC_CONFIG_MACRO_DIR, and that in turn causes
> aclocal.m4 to be out of date (because it needs to be regenerated to pick
> up the local versions of the libtoolize added m4 files added to
> ../config/ after it was first generated).

actually, (at least modern enough) autoreconf runs the aclocal twice.
Once before libtoolize call (do detect whether it should call the
libtoolize tool at all) and second time [1] after libtoolize to
incorporate the macros.

I'd like to say in advance that IMO this should be fixed in libtoolize,
somehow, reasoning follows.

- autoreconf is really generally believed to be the "tool #1" for
  regenerating GNU buildsystem .. if we don't take the 'gnulib' or others
  into account of course;  then however bootstrap comes and autoreconf
  -vfi should still be OK

- thus either libtool or autoreconf should be fixed, ..

- when libltdl is included into project as 'convenience' library, it is
  treated like "subproject".  Then running autoreconf from project's
  $(top_srcdir) treats subprojects independently by recursing down into
  subdirectory (in our case libltdl) and autoreconfing there.

  The child autoreconf does not expect that something defined in parent
  directory will touch subdirectories after successful autoreconf.

  When parent autoreconf finished with autoreconfing of subdirectory, it
  continues in cwd - and that includes running libtoolize; _however_, this
  second run of libtoolize from top-level overwrites the _common_ macros.
  Now, the macros are more recent than the "subdirectory thinks".

  After that ^^^, the aclocal tool is run in top-level directory - but it
  has no idea about recursing down into (otherwise independent)
  subdirectory.  So everything happens as expected, two separate
  directories were autoreconfed independently.

> The bootstrap script in the libtool source tree fixes this (and many
> other problems with the autogen.sh/autoreconf approach), so if you care
> to write a bootstrap.conf (by copying and hacking nearly everything out
> of libtool-2.4.4/bootstrap.conf), things are then created in the right
> order and the bug disappears.
> 
> Alternatively, you can amend your autogen.sh to something like this:
> 
>   libtoolize --install --copy --ltdl
>   LIBTOOLIZE=true autoreconf -fvi

Yes, this work-arounds that.  Thanks!

> If it worked for you in 2.4.2 in that order, then it was just a lucky
> combination of an empty local directory and installed versions of the
> macro files in the right place for aclocal.m4 to be valid on the initial
> too-early run.

Hmm, I think sharing the macros with top-level is clear trigger here.

> In your original report, however, you said:
> 
> "The problem appears to be that make is checking for ../m4/libtool.m4
> file as a dependency.  This file file -- and the entire ../m4 directory,
> for that matter -- does not exist.  So make decides to fire the "run the
> aclocal" rule."
>
> ...which seems odd to me, because for a subproject libltdl, the parent
> AC_CONFIG_MACRO_DIR/ACLOCAL_AMFLAGS directory is supposed to be merged
> in.  Did you mean to say "../config/libtool.m4" above?  If that
> substitution really isn't happening, then you've found a different bug -
> but I can't reproduce that one with 2.4.3, 2.4.4 nor current master.

Reproduced.  I'll try to make automatized reproducer and post later
possibly.

To make it clear:  The top-level libtoolize re-initializes the
sub-directory macros without runing autoreconf.  Makefile.in _is already_
generated (distributed via libtool) and put on place.  The Makefile.in
distributed with libtool contains:

  am__aclocal_m4_deps = $(top_srcdir)/../m4/libtool.m4 \
  >-------$(top_srcdir)/../m4/ltargz.m4 $(top_srcdir)/../m4/ltdl.m4 \
  >-------$(top_srcdir)/../m4/ltoptions.m4 \
  >-------$(top_srcdir)/../m4/ltsugar.m4 \
  >-------$(top_srcdir)/../m4/ltversion.m4 \
  >-------$(top_srcdir)/../m4/lt~obsolete.m4 $(top_srcdir)/configure.ac

Without regenerating Makefile.in, bad things happen if user has
non-default AC_CONFIG_MACRO_DIR.  Also, proposed fix [2] is kind of
related.

-------

For the solution, I was thinking about something like detecting whether
libtoolize is run from $(srcdir) or not.  Something like:

  --- a/libtoolize.in
  +++ b/libtoolize.in
  @@ -898,6 +898,8 @@ func_install_pkgltdl_files ()
       $require_ltdl_dir
       $require_ltdl_mode

  +    test '.' = "$ltdl_dir" || return
  +
       # Remove any lingering files that my have been installed by some
       # previous libtoolize release:
       $opt_force && for file in $all_pkgltdl_files; do

.. or turning maintainer mode on in libltdl?  Documenting autoreconf-ing by
hand in subdirectory is also an option (after top-dir is done) but that is
counter-intuitive without 'autoreconf --no-recursive'.

That all because it seems to me that hacking this in autoreconf is (a) imo
too difficult and (b) too late as libtool-2.4.4 is out.

[1] 
http://git.savannah.gnu.org/cgit/autoconf.git/tree/bin/autoreconf.in?id=7b13e39a112309786ebb2fdb76e027b7eaa4f2f5#n486
[2] http://news.gmane.org/gmane.comp.gnu.libtool.patches

Pavel