Re: Documentation for conditional SUBDIRS (+ test cases)

[Alexandre Duret-Lutz]

>>> "Bruce" == Bruce Korb <address@hidden> writes:

[...]

 Bruce> My comments below. 

Thanks a lot!

 Bruce> Can I have comments vis-a-vis:

 Bruce> http://autogen.sf.net/blocksort.html  :-)

I'll send you a separate mail.

[...]

 >> The `SUBDIRS' variable holds a list of subdirectories in which
 >> building of various sorts can occur.

 Bruce> CAVEAT:  the list must *not* include the AC_CONFIG_AUX_DIR() directory,
 Bruce> if used.  

I've always done this.  Many packages do this.

[...]

 Bruce> There *is* a problem, but I guess I should isolate which
 Bruce> it is.

Please.  Off-hand I'd vote for a consequence of that EXTRA_DIST trickery.

 >> [[...]]  The directories mentioned in
 >> `SUBDIRS' must be direct children of the current directory.  For
 >> instance, you cannot put `src/subdir' into `SUBDIRS'.

 Bruce> << at this point I would like to see either mention of how to handle
 Bruce> sub-subdirectories, or else a link to such a discussion. >>

 >> The use of `SUBDIRS' is not restricted to just the top-level
 >> `Makefile.am'.  Automake can be used to construct packages of arbitrary
 >> depth.

 Bruce> << now I'm confused. >>

Ok, I've merged these two paragraphes into

|    The directories mentioned in `SUBDIRS' must be direct children of
| the current directory.  For instance, you cannot put `src/subdir' into
| `SUBDIRS'.  Instead you should put `SUBDIRS = subdir' into
| `src/Makefile.am'.  Automake can be used to construct packages of
| arbitrary depth this way.


[...]

 Bruce> << It sounds like we need a table of standard targets and which
 Bruce> SUBDIRS variable gets used, and a discussion of how DIST_SUBDIRS
 Bruce> gets derived >>

Let's make this a fourth section.

| How `DIST_SUBDIRS' is used
| --------------------------
|    
|    As shown in the above examples, `DIST_SUBDIRS' is used by targets
| that need to recurse in all directories, even those which have been
| conditionally left out of the build.
| 
|    Precisely, `DIST_SUBDIRS' is used by `make dist', `make distclean',
| and `make maintainer-clean'.  All other recursive targets use `SUBDIRS'.
|    
|    Automake will define `DIST_SUBDIRS' automatically from the possibles
| values of `SUBDIRS' in all conditions.
| 
|    If `SUBDIRS' contains `AC_SUBST' variables, `DIST_SUBDIRS' will not
| be defined correctly because Automake doesn't know the possible values
| of these variables.  In this case `DIST_SUBDIRS' needs to be defined
| manually.

Here is the patch I'm checking in.

2002-08-23  Alexandre Duret-Lutz  <address@hidden>

        * automake.texi (Top level): More words about conditional
        subdirectories.  Don't mention AC_PROG_MAKE_SET.
        * tests/subdircond2.test, tests/subdircond3.test: New files.
        * tests/Makefile.am (TESTS): Add them.

Index: automake.texi
===================================================================
RCS file: /cvs/automake/automake/automake.texi,v
retrieving revision 1.297
diff -u -r1.297 automake.texi
--- automake.texi       22 Aug 2002 17:23:03 -0000      1.297
+++ automake.texi       23 Aug 2002 19:09:04 -0000
@@ -1756,6 +1756,8 @@
 @node Top level, Alternative, configure, Top
 @chapter The top-level @file{Makefile.am}
 
address@hidden Recursing subdirectories
+
 @cindex SUBDIRS, explained
 
 In packages with subdirectories, the top level @file{Makefile.am} must
@@ -1769,10 +1771,7 @@
 subdirectories.  Note that the directories listed in @code{SUBDIRS} are
 not required to contain @file{Makefile.am}s; only @file{Makefile}s
 (after configuration).  This allows inclusion of libraries from packages
-which do not use Automake (such as @code{gettext}).  The directories
-mentioned in @code{SUBDIRS} must be direct children of the current
-directory.  For instance, you cannot put @samp{src/subdir} into
address@hidden
+which do not use Automake (such as @code{gettext}).
 
 In packages that use subdirectories, the top-level @file{Makefile.am} is
 often very short.  For instance, here is the @file{Makefile.am} from the
@@ -1783,63 +1782,152 @@
 SUBDIRS = doc intl po src tests
 @end example
 
address@hidden SUBDIRS, overriding
address@hidden Overriding SUBDIRS
+When Automake invokes @code{make} in a subdirectory, it uses the value
+of the @code{MAKE} variable.  It passes the value of the variable
address@hidden to the @code{make} invocation; this can be set in
address@hidden if there are flags you must always pass to
address@hidden
address@hidden MAKE
address@hidden MAKEFLAGS
+
+The directories mentioned in @code{SUBDIRS} must be direct children of
+the current directory.  For instance, you cannot put @samp{src/subdir}
+into @code{SUBDIRS}.  Instead you should put @code{SUBDIRS = subdir}
+into @file{src/Makefile.am}.  Automake can be used to construct packages
+of arbitrary depth this way.
+
+By default, Automake generates @file{Makefiles} which work depth-first
+(@samp{postfix}).  However, it is possible to change this ordering.  You
+can do this by putting @samp{.} into @code{SUBDIRS}.  For instance,
+putting @samp{.}  first will cause a @samp{prefix} ordering of
+directories.  All @samp{clean} targets are run in reverse order of build
+targets.
+
address@hidden Conditional subdirectories
address@hidden Subdirectories, building conditionally
address@hidden Conditional subdirectories
address@hidden @code{SUBDIRS}, conditional
address@hidden Conditional @code{SUBDIRS}
+
+It is possible to define the @code{SUBDIRS} variable conditionally if,
+like in the case of GNU @code{Inetutils}, you want to only build a
+subset of the entire package.
+
+To illustrate how this works, let's assume we have two directories
address@hidden/} and @file{opt/}.  @file{src/} should always be built, but we
+want to decide in @code{./configure} whether @file{opt/} will be built
+or not.  (For this example we will assume that @file{opt/} should be
+built when the variable @code{$want_opt} was set to @code{yes}.)
+
+Running @code{make} should thus recurse into @file{src/} always, and
+then maybe in @file{opt/}.
+
+However @code{make dist} should always recurse into both @file{src/} and
address@hidden/}.  Because @file{opt/} should be distributed even if it is
+not needed in the current configuration. This means @file{opt/Makefile}
+should be created unconditionally.  @footnote{Don't try seeking a
+solution where @file{opt/Makefile} is created conditionally, this is a
+lot trickier than the solutions presented here.}
+
+There are two ways to setup a project like this.  You can use Automake
+conditionals (@pxref{Conditionals}) or use Autoconf @code{AC_SUBST}
+variables (@pxref{Setting Output Variables, , Setting Output Variables,
+autoconf, The Autoconf Manual}).  Using Automake conditionals is the
+preferred solution.
+
address@hidden Conditional subdirectories with @code{AM_CONDITIONAL}
address@hidden @code{SUBDIRS} and @code{AM_CONDITIONAL}
address@hidden @code{AM_CONDITIONAL} and @code{SUBDIRS}
+
address@hidden The test case for the setup described here is
address@hidden     test/subdircond2.test
address@hidden Try to keep it in sync.
 
-It is possible to override the @code{SUBDIRS} variable if, like in the
-case of GNU @code{Inetutils}, you want to only build a subset of the
-entire package.  In your @file{Makefile.am} include:
address@hidden should output the @file{Makefile} for each directory
+and define a condition into which @file{opt/} should be built.
 
 @example
-SUBDIRS = @@MY_SUBDIRS@@
address@hidden
+AM_CONDITIONAL([COND_OPT], [test "$want_opt" = yes])
+AC_CONFIG_FILES([Makefile src/Makefile opt/Makefile])
address@hidden
 @end example
 
-Then in your @file{configure.in} you can specify:
+Then @code{SUBDIRS} can be defined in the top-level @file{Makefile.am}
+as follows.
 
 @example
-MY_SUBDIRS="src doc lib po"
-AC_SUBST(MY_SUBDIRS)
+if COND_OPT
+  MAYBE_OPT = opt
+endif
+SUBDIRS = src $(MAYBE_OPT)
 @end example
 
-(Note that we don't use the variable name @code{SUBDIRS} in our
address@hidden; that would cause Automake to believe that every
address@hidden should recurse into the listed subdirectories.)
-
-The upshot of this is that Automake is tricked into building the package
-to take the subdirs, but doesn't actually bind that list until
address@hidden is run.
-
-Although the @code{SUBDIRS} variable can contain configure substitutions
-(e.g. @samp{@@DIRS@@}); Automake itself does not actually examine the
-contents of this variable.
-
-If @code{SUBDIRS} is defined, then your @file{configure.in} must include
address@hidden  When Automake invokes @code{make} in a
-subdirectory, it uses the value of the @code{MAKE} variable.  It passes
-the value of the variable @code{AM_MAKEFLAGS} to the @code{make}
-invocation; this can be set in @file{Makefile.am} if there are flags you
-must always pass to @code{make}.
address@hidden MAKE
address@hidden MAKEFLAGS
+As you can see, running @code{make} will rightly recurse into
address@hidden/} and maybe @file{opt/}.
 
-The use of @code{SUBDIRS} is not restricted to just the top-level
address@hidden  Automake can be used to construct packages of
-arbitrary depth.
address@hidden DIST_SUBDIRS
+As you can't see, running @code{make dist} will recurse into both
address@hidden/} and @file{opt/} directories because @code{make dist}, unlike
address@hidden all}, doesn't use the @code{SUBDIRS} variable.  It uses the
address@hidden variable.
+
+In this case Automake will define @code{DIST_SUBDIRS = src opt}
+automatically because it knows that @code{MAYBE_OPT} can contain
address@hidden in some condition.
+
address@hidden Conditional subdirectories with @code{AC_SUBST}
address@hidden @code{SUBDIRS} and @code{AC_SUBST}
address@hidden @code{AC_SUBST} and @code{SUBDIRS}
+
address@hidden The test case for the setup described here is
address@hidden     test/subdircond3.test
address@hidden Try to keep it in sync.
 
-By default, Automake generates @file{Makefiles} which work depth-first
-(@samp{postfix}).  However, it is possible to change this ordering.  You
-can do this by putting @samp{.} into @code{SUBDIRS}.  For instance,
-putting @samp{.}  first will cause a @samp{prefix} ordering of
-directories.  All @samp{clean} targets are run in reverse order of build
-targets.
+Another idea is to define @code{MAYBE_OPT} from @file{./configure} using
address@hidden:
+
address@hidden
address@hidden
+if test "$want_opt" = yes; then
+  MAYBE_OPT=opt
+else
+  MAYBE_OPT=
+fi
+AC_SUBST([MAYBE_OPT])
+AC_CONFIG_FILES([Makefile src/Makefile opt/Makefile])
address@hidden
address@hidden example
+
+In this case the top-level @file{Makefile.am} should look as follows.
+
address@hidden
+SUBDIRS = src $(MAYBE_OPT)
+DIST_SUBDIRS = src opt
address@hidden example
 
-Sometimes, such as when running @code{make dist}, you want all possible
-subdirectories to be examined.  In this case Automake will use
address@hidden, instead of @code{SUBDIRS}, to determine where to
-recurse.  This variable will also be used when the user runs
address@hidden or @code{maintainer-clean}.  It should be set to the
-full list of subdirectories in the project.  If this variable is not set,
-Automake will attempt to set it for you.
+The drawback is that since Automake cannot guess what the possible
+values of @code{MAYBE_OPT} are, it is necessary to define
address@hidden
+
address@hidden How @code{DIST_SUBDIRS} is used
address@hidden @code{DIST_SUBDIRS}, explained
+
+As shown in the above examples, @code{DIST_SUBDIRS} is used by targets
+that need to recurse in all directories, even those which have been
+conditionally left out of the build.
+
+Precisely, @code{DIST_SUBDIRS} is used by @code{make dist}, @code{make
+distclean}, and @code{make maintainer-clean}.  All other recursive
+targets use @code{SUBDIRS}.
+
+Automake will define @code{DIST_SUBDIRS} automatically from the
+possibles values of @code{SUBDIRS} in all conditions.
+
+If @code{SUBDIRS} contains @code{AC_SUBST} variables,
address@hidden will not be defined correctly because Automake
+doesn't know the possible values of these variables.  In this case
address@hidden needs to be defined manually.
 
 
 @node Alternative, Rebuilding, Top level, Top
Index: tests/Makefile.am
===================================================================
RCS file: /cvs/automake/automake/tests/Makefile.am,v
retrieving revision 1.430
diff -u -r1.430 Makefile.am
--- tests/Makefile.am   23 Aug 2002 13:32:55 -0000      1.430
+++ tests/Makefile.am   23 Aug 2002 19:09:04 -0000
@@ -345,6 +345,8 @@
 subdir7.test \
 subdirbuiltsources.test \
 subdircond.test \
+subdircond2.test \
+subdircond3.test \
 subobj.test \
 subobj2.test \
 subobj3.test \
Index: tests/subdircond2.test
===================================================================
RCS file: tests/subdircond2.test
diff -N tests/subdircond2.test
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ tests/subdircond2.test      23 Aug 2002 19:09:05 -0000
@@ -0,0 +1,77 @@
+#! /bin/sh
+
+# The for conditional SUBDIRS.
+# SUBDIRS + AM_CONDITIONAL setup from the manual.
+# Lots of lines here are duplicated in subdircond3.test.
+
+. $srcdir/defs || exit 1
+
+set -e
+
+cat >>configure.in <<'END'
+AM_CONDITIONAL([COND_OPT], [test "$want_opt" = yes])
+AC_CONFIG_FILES([src/Makefile opt/Makefile])
+AC_OUTPUT
+END
+
+cat >Makefile.am << 'END'
+if COND_OPT
+  MAYBE_OPT = opt
+endif
+SUBDIRS = src $(MAYBE_OPT)
+
+# Testing targets.
+#
+# We want to ensure that
+#      - src/source and opt/source are always distributed.
+#      - src/result is always built
+#      - opt/result is built conditionally
+#
+# We rely on `distcheck' to run `check-local' and use
+# `sanity1' and `sanity2' as evidences that test-build was run.
+
+if COND_OPT
+test-build: all
+       test -f src/result
+       test -f opt/result
+       : > $(top_builddir)/../../sanity2
+else
+test-build: all
+       test -f src/result
+       test ! -f opt/result
+       : > $(top_builddir)/../../sanity1
+endif
+
+test-dist: distdir
+       test -f $(distdir)/src/source
+       test -f $(distdir)/opt/source
+
+check-local: test-build test-dist
+END
+
+mkdir src opt
+: > src/source
+: > opt/source
+
+cat >src/Makefile.am << 'END'
+EXTRA_DIST = source
+all-local: result
+CLEANFILES = result
+
+result: source
+       cp $(srcdir)/source result
+END
+
+# We want in opt/ the same Makefile as in src/.  Let's exercize `include'.
+cat >opt/Makefile.am << 'END'
+include ../src/Makefile.am
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+./configure
+$MAKE distcheck
+test -f sanity1
+$MAKE DISTCHECK_CONFIGURE_FLAGS=want_opt=yes distcheck
+test -f sanity2
Index: tests/subdircond3.test
===================================================================
RCS file: tests/subdircond3.test
diff -N tests/subdircond3.test
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ tests/subdircond3.test      23 Aug 2002 19:09:05 -0000
@@ -0,0 +1,78 @@
+#! /bin/sh
+
+# The for conditional SUBDIRS.
+# SUBDIRS + AC_SUBST setup from the manual.
+# Lots of lines here are duplicated in subdircond2.test.
+
+. $srcdir/defs || exit 1
+
+set -e
+
+cat >>configure.in <<'END'
+if test "$want_opt" = yes; then
+  MAYBE_OPT=opt
+else
+  MAYBE_OPT=
+fi
+AC_SUBST([MAYBE_OPT])
+AC_CONFIG_FILES([src/Makefile opt/Makefile])
+AC_OUTPUT
+END
+
+cat >Makefile.am << 'END'
+SUBDIRS = src $(MAYBE_OPT)
+DIST_SUBDIRS = src opt
+
+# Testing targets.
+#
+# We want to ensure that
+#      - src/source and opt/source are always distributed.
+#      - src/result is always built
+#      - opt/result is built conditionally
+#
+# We rely on `distcheck' to run `check-local' and use
+# `sanity1' and `sanity2' as evidences that test-build was run.
+
+test-build: all
+       test -f src/result
+       if test -n "$(MAYBE_OPT)"; then \
+          test -f opt/result || exit 1; \
+          : > $(top_builddir)/../../sanity2 || exit 1; \
+       else \
+          test ! -f opt/result || exit 1; \
+          : > $(top_builddir)/../../sanity1 || exit 1; \
+       fi
+
+test-dist: distdir
+       test -f $(distdir)/src/source
+       test -f $(distdir)/opt/source
+
+check-local: test-build test-dist
+END
+
+mkdir src opt
+: > src/source
+: > opt/source
+
+cat >src/Makefile.am << 'END'
+EXTRA_DIST = source
+all-local: result
+CLEANFILES = result
+
+result: source
+       cp $(srcdir)/source result
+END
+
+# We want in opt/ the same Makefile as in src/.  Let's exercize `include'.
+cat >opt/Makefile.am << 'END'
+include ../src/Makefile.am
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+./configure
+$MAKE distcheck
+test -f sanity1
+$MAKE DISTCHECK_CONFIGURE_FLAGS=want_opt=yes distcheck
+test -f sanity2

-- 
Alexandre Duret-Lutz