Fixing parallel install of libtool libs and its dependents

[Ralf Wildenhues]

A long-standing issue in Automake:  Fixing the ordering issue of
"make install" and maybe also parallel "make install" when shared
(in the context of Automake: libtool-created) libraries are in use.

First some history (please post more links if you find additional
useful bits of information):
Nice long explanation and thread about the issue:
http://lists.gnu.org/archive/html/libtool/2004-10/msg00052.html
more bug reports:
http://lists.gnu.org/archive/html/automake/2004-07/msg00125.html
http://lists.gnu.org/archive/html/automake/2004-03/msg00090.html
http://lists.gnu.org/archive/html/automake/2004-01/msg00145.html
automake/TODO has an entry;
Halfway fix of original "prog -> ltlib" problem:
http://www.mail-archive.com/address@hidden/msg00624.html


My idea is the following: I believe the order problem can be solved
in Automake alone, with `make' doing the topological sort.

The example Makefile (in the manner which Automake should create)
below introduces a new suffix rule which turns the libtool libraries
into installed libtool libraries.  Actual new stuff is marked NEW.

Notes:
======
- This approach assumes that multi-Makefile projects will have the
corresponding order guaranteed by SUBDIRS ordering.  This also means
that at the time `install' is targeted by make, all .la files have
been built.  Is this a sensible assumption?

- Conditionals should be straightforward (I believe).

- Configure-time substitutions cause trouble with libtool libraries
even before this patch (because of possibly missing -rpath information).
I have still tried to cope with them in the example below by using
another $(MAKE) indirection.  See at end for possible simplifications.

- if more than one prefix are used in conjunction with the primary
LTLIBRARIES, they could be combined as indicated in comments.

- circular dependencies will break this approach completely.  Right
now I don't know a better solution than to turn off this approach in
that case.  IOW: enable this whole thingy through an Automake option.

- The actual recording of inter-library dependencies might be done in
automake (if we can skip substitutions), or with help of libtool.
They might live in subdirectory files, similar to $(DEPDIR).

- the .la-inst targets are not marked phony.  IMHO a small issue.

- tested with all make programs I could find on GNU/Linux, Solaris,
FreeBSD, AIX (actual source dependencies for the .la files might have
to be added so that the example Makefile below works ok).


Do you think this could be a feasible approach?  Are there any
portability issues I have overlooked?  Would this be a worthwhile
goal for Automake or overkill?  Do you have a better idea for
circular dependencies?

I'd like to know whether I'm wrong before trying to understand
Automake innards.  Also I'd need someone to either code this for me
or help me understand /usr/bin/automake a lot better than I do now.

Cheers,
Ralf

### snip Makefile
# common variables
am__strip_dir = `echo $$p | sed -e 's|^.*/||'`;
libdir = prefix/lib
libLTLIBRARIES_INSTALL = install
mkdir_p = echo mkdir -p
INSTALL_STRIP_FLAG = 
LIBTOOL = libtool

# default target for this test
install-exec-am: install-libLTLIBRARIES

# next line user-supplied: substituted values do not work here.
lib_LTLIBRARIES = libFOO.la libBAR.la libBAZ.la sub/libSUB.la

# stubs for ltlib creation
libFOO.la:
        @echo build $@
        @sleep 1
        @echo build $@ done
libBAR.la:
        @echo build $@
        sleep 1
        @echo build $@ done
libBAZ.la:
        @echo build $@
        sleep 1
        @echo build $@ done
sub/libSUB.la:
        @echo build $@
        sleep 1
        @echo build $@ done

# NEW: install rule (here only for `lib' prefix)
# The `make' reinvocation is necessary to cope with substitutions.
install-libLTLIBRARIES:
        test -z "$(libdir)" || $(mkdir_p) "$(DESTDIR)$(libdir)"
        @list=`echo $(lib_LTLIBRARIES) | sed 's/\.la/&-inst/g'`; \
        $(MAKE) $(AM_MAKEFLAGS) $$list

# For more than one primary, above rule should instead look like this:
# install-libLTLIBRARIES install-pkglibLTLIBRARIES:
#       test -z "$(libdir)" || $(mkdir_p) "$(DESTDIR)$(libdir)"
#       test -z "$(pkglibdir)" || $(mkdir_p) "$(DESTDIR)$(pkglibdir)"
#       @list=`echo $(lib_LTLIBRARIES) $(pkglib_LTLIBRARIES) | sed 
's/\.la/&-inst/g'`; \
#       $(MAKE) $(AM_MAKEFLAGS) $$list

# NEW suffixes and suffix rule
# TODO: What do we do with libraries not built from this Makefile?
# Does the `case' work around this sufficiently?  The .la files should
# not be rewritten, they are in the build tree only.
.SUFFIXES: .la .la-inst
.la.la-inst:
        @echo inst $@; sleep 1
        @p='$<'; f=$(am__strip_dir) \
        case " $$p " in " $(lib_LTLIBRARIES) ") \
          echo $(LIBTOOL) --mode=install $(libLTLIBRARIES_INSTALL) 
$(INSTALL_STRIP_FLAG) "$$p" "$(DESTDIR)$(libdir)/$$f";; esac
        @echo inst $@ done
# For more than one primary, add to this suffix rule:
#       case " $$p " in " $(pkglib_LTLIBRARIES) ") \
#         echo $(LIBTOOL) --mode=install $(pkglibLTLIBRARIES_INSTALL) 
$(INSTALL_STRIP_FLAG) "$$p" "$(DESTDIR)$(pkglibdir)/$$f";; esac
#       ...

# NEW: actual library dependencies
# Can these be created at automake time?  Not if substitutions allowed.
# Can libtool help here at make time?  Yes.
# With libs not built from this Makefile, we should be fine..
# (add dummy line `libFOO.la-inst:' if necessary).
libFOO.la-inst: libBAR.la-inst
libBAR.la-inst: libBAZ.la-inst
libBAZ.la-inst: sub/libSUB.la-inst
# Can we also get individual program dependencies?
# Would this be sufficient instead?
# install_binPROGRAMS: libfoo.la-inst [...]
### snip



If we could ignore substitutions in *_LTLIBRARIES (for which there
are good reasons ATM), then we could kill the `make' reinvocation,
with rules like this instead:

# NEW: next line computed by Automake.
inst_lib_LTLIBRARIES = libFOO.la-inst libBAR.la-inst libBAZ.la-inst 
sub/libSUB.la-inst
# inst_pkglib_LTLIBRARIES = libbla.la-pkginst ...

# NEW: all depend on existence of the installation directory
$(inst_lib_LTLIBRARIES): inst_lib_LTLIBRARIES_makedir
# $(inst_pkglib_LTLIBRARIES): inst_pkglib_LTLIBRARIES_makedir

# NEW: install dir creation rule
# It should be sufficient to have this as phony rule -- it will require
# all other installs to be recreated.
inst_lib_LTLIBRARIES_makedir:
        test -z "$(libdir)" || $(mkdir_p) "$(DESTDIR)$(libdir)"
.PHONY: inst_lib_LTLIBRARIES_makedir

# NEW: depend on the individual install targets.
install-libLTLIBRARIES: $(inst_lib_LTLIBRARIES)