Re: moving $(mkdir_p) from automake to autoconf

[Paul Eggert]

Paul Eggert <address@hidden> writes:

> I think my worry is a bit more general than that, in that
> AC_PROG_MKDIR_P requires AC_PROG_INSTALL,

I looked into this a bit more and decided that it would be safer to
back out my change to AC_PROG_INSTALL, and for AC_PROG_MKDIR_P to fall
back to install-sh (rather than to $INSTALL) when mkdir -p isn't known
to be thread-safe.  If I hadn't done this, I would have had to modify
AC_PROG_INSTALL to check that install -d was thread-safe, and this
would have been a bigger change.

Also, I changed AC_PROG_MKDIR_P to set MKDIR_P, not mkdir_p, to avoid
collision with Automake.  This also should be safer.

I installed the following patch, which I hope addresses all the
comments made in this thread so far, plus a couple of other things
that I noticed.

2006-05-13  Paul Eggert  <address@hidden>

        * doc/autoconf.texi (Particular Programs): AC_PROG_MKDIR_P now
        sets MKDIR_P, not mkdir_p, to avoid collisions with Automake.
        Warn about obsolete install-sh files.  Remove stray sentence
        fragment and fix cross reference.
        * lib/autoconf/programs.m4 (AC_PROG_INSTALL): Don't insist on
        install -d; this undoes the 2006-05-10 change.
        (MKDIR_P): Mark with AN_MAKEVAR.
        (AC_PROG_MKDIR_P): Fall back on $ac_install_sh, not $INSTALL, so
        that we don't require $INSTALL to be thread-safe.  Move comments
        out of generated code.  Require AC_CONFIG_AUX_DIR_DEFAULT instead
        of AC_PROG_INSTALL.  Output a message saying that we're checking
        mkdir -p.  Set MKDIR_P rather than mkdir_p.  Do special magic for
        MKDIR_P instead of AC_SUBST.
        * lib/autoconf/status.m4 (_AC_OUTPUT_FILE, _AC_OUTPUT_CONFIG_STATUS):
        Special magic for MKDIR_P, too.
        * lib/m4sugar/m4sh.m4 (AS_MKDIR_P): Remove comment that defeated
        a dnl.
        * tests/local.at (AT_CHECK_ENV): mkdir_p -> MKDIR_P.

Index: doc/autoconf.texi
===================================================================
RCS file: /cvsroot/autoconf/autoconf/doc/autoconf.texi,v
retrieving revision 1.1016
diff -p -u -r1.1016 autoconf.texi
--- doc/autoconf.texi   10 May 2006 23:42:59 -0000      1.1016
+++ doc/autoconf.texi   13 May 2006 09:51:41 -0000
@@ -1824,7 +1824,7 @@ Invocation}, for more information.
 @c just like any text that appear before @code{AC_INIT}.
 @c @end defmac
 
-Packages that do manual configuration or use the @code{install} program
+Packages that do manual configuration or use the @command{install} program
 might need to tell @command{configure} where to find some other shell
 scripts by calling @code{AC_CONFIG_AUX_DIR}, though the default places
 it looks are correct for most cases.
@@ -3493,14 +3493,14 @@ lines.
 @ovindex INSTALL_DATA
 @ovindex INSTALL_SCRIPT
 Set output variable @code{INSTALL} to the name of a @acronym{BSD}-compatible
address@hidden program, if one is found in the current @env{PATH}.
address@hidden program, if one is found in the current @env{PATH}.
 Otherwise, set @code{INSTALL} to @address@hidden/install-sh -c},
 checking the directories specified to @code{AC_CONFIG_AUX_DIR} (or its
 default directories) to determine @var{dir} (@pxref{Output}).  Also set
 the variables @code{INSTALL_PROGRAM} and @code{INSTALL_SCRIPT} to
 @address@hidden@}} and @code{INSTALL_DATA} to @address@hidden@} -m 644}.
 
-This macro screens out various instances of @code{install} known not to
+This macro screens out various instances of @command{install} known not to
 work.  It prefers to find a C program rather than a shell script, for
 speed.  Instead of @file{install-sh}, it can also use @file{install.sh},
 but that name is obsolete because some @command{make} programs have a rule
@@ -3510,40 +3510,44 @@ Autoconf comes with a copy of @file{inst
 you use @code{AC_PROG_INSTALL}, you must include either
 @file{install-sh} or @file{install.sh} in your distribution, or
 @command{configure} will produce an error message saying it can't find
-them---even if the system you're on has a good @code{install} program.
+them---even if the system you're on has a good @command{install} program.
 This check is a safety measure to prevent you from accidentally leaving
 that file out, which would prevent your package from installing on
-systems that don't have a @acronym{BSD}-compatible @code{install} program.
+systems that don't have a @acronym{BSD}-compatible @command{install} program.
 
 If you need to use your own installation program because it has features
-not found in standard @code{install} programs, there is no reason to use
+not found in standard @command{install} programs, there is no reason to use
 @code{AC_PROG_INSTALL}; just put the file name of your program into your
 @file{Makefile.in} files.
 @end defmac
 
 @defmac AC_PROG_MKDIR_P
 @acindex{AC_PROG_MKDIR_P}
address@hidden mkdir_p
-Set output variable @code{mkdir_p} to a command that ensures that for
address@hidden MKDIR_P
+Set output variable @code{MKDIR_P} to a program that ensures that for
 each argument, a directory named by this argument exists, creating it
-and its parent directories if needed.  The command is checked to make
+and its parent directories if needed.  The program is checked to make
 sure that it is thread-safe (@pxref{Limitations of Usual Tools}).
 
 This macro uses the @samp{mkdir -p} command if possible.  Otherwise, it
-falls back on @code{$INSTALL}, so your package should
+falls back on invoking @command{install-sh} with the @option{-d} option,
+so your package should
 contain @file{install-sh} as described under @code{AC_PROG_INSTALL}.
+A @file{install-sh} file that predates Autoconf 2.60 or Automake 1.10
+won't be thread-safe, so if you want to support parallel installs from
+different packages into the same directory you need to make sure you
+have an up-to-date @file{install-sh}.  In particular, be careful about
+using @samp{autoreconf -if} if your Automake predates Automake 1.10.
 
 This macro is related to the @code{AS_MKDIR_P} macro (@pxref{Programming
 in M4sh}), but it sets an output variable intended for use in other
 files, whereas @code{AS_MKDIR_P} is intended for use in scripts like
 @command{configure}.  Also, @code{AS_MKDIR_P} does not accept options,
 but if you are willing to assume Posix 1003.2-1992 or later, a
address@hidden can use options, e.g., a makefile might invoke
address@hidden(mkdir_p) -m 0 dir}.
address@hidden can use the @option{-m} option, e.g., a makefile might
+invoke @code{$(MKDIR_P) -m 0 dir} to create an inaccessible directory.
 @end defmac
 
-This macro differs from
-
 @defmac AC_PROG_LEX
 @acindex{PROG_LEX}
 @ovindex LEX
@@ -9750,8 +9754,7 @@ even though Posix is unclear whether @sa
 succeed in that case.  If creation of @var{file-name} fails, exit the
 script.
 
-Also see the @code{AC_PROG_MKDIR_P} macro (@pxref{Limitations of Usual
-Tools}).
+Also see the @code{AC_PROG_MKDIR_P} macro (@pxref{Particular Programs}).
 @end defmac
 
 @defmac AS_SHELL_SANITIZE
@@ -13082,8 +13085,8 @@ race-free @code{mkdir -p}.  This possibl
 builds when several @file{Makefile} rules call @code{mkdir -p} to
 construct directories.  You may use
 @code{install-sh -d} as a safe replacement, provided this script is
-recent enough (the copy shipped with Autoconf 2.60 is OK, those from
-older versions are not thread-safe either).
+recent enough; the copy shipped with Autoconf 2.60 and Automake 1.10 is
+OK, but copies from older versions are not thread-safe either.
 
 
 @item @command{mktemp}
Index: lib/autoconf/programs.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/autoconf/programs.m4,v
retrieving revision 1.52
diff -p -u -r1.52 programs.m4
--- lib/autoconf/programs.m4    10 May 2006 23:43:36 -0000      1.52
+++ lib/autoconf/programs.m4    13 May 2006 09:51:42 -0000
@@ -568,16 +568,13 @@ case $as_dir/ in
            grep pwplus "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then
            # program-specific install script used by HP pwplus--don't use.
            :
-         elif rm -fr conftest.dir &&
-           $as_dir/$ac_prog$ac_exec_ext -c -d conftest.dir/d >/dev/null 2>&1 &&
-           rmdir conftest.dir/d conftest.dir >/dev/null 2>&1; then
+         else
            ac_cv_path_install="$as_dir/$ac_prog$ac_exec_ext -c"
            break 3
          fi
        fi
       done
     done
-    rm -fr conftest.dir
     ;;
 esac])
 ])dnl
@@ -610,7 +607,8 @@ AC_SUBST(INSTALL_DATA)dnl
 
 # AC_PROG_MKDIR_P
 # ---------------
-# Check whether `mkdir -p' is supported, fallback to mkinstalldirs otherwise.
+# Check whether `mkdir -p' is known to be thread-safe, and fall back to
+# install-sh -d otherwise.
 #
 # Automake 1.8 used `mkdir -m 0755 -p --' to ensure that directories
 # created by `make install' are always world readable, even if the
@@ -630,40 +628,47 @@ AC_SUBST(INSTALL_DATA)dnl
 # one can create it and the other will error out.  Consequently we
 # restrict ourselves to GNU mkdir (using the --version option ensures
 # this.)
+#
+# Automake used to define mkdir_p as `mkdir -p .', in order to
+# allow $(mkdir_p) to be used without argument.  As in
+#   $(mkdir_p) $(somedir)
+# where $(somedir) is conditionally defined.  However we don't do
+# that for MKDIR_P.
+#  1. before we restricted the check to GNU mkdir, `mkdir -p .' was
+#     reported to fail in read-only directories.  The system where this
+#     happened has been forgotten.
+#  2. in practice we call $(MKDIR_P) on directories such as
+#       $(MKDIR_P) "$(DESTDIR)$(somedir)"
+#     and we don't want to create $(DESTDIR) if $(somedir) is empty.
+#     To support the latter case, we have to write
+#       test -z "$(somedir)" || $(MKDIR_P) "$(DESTDIR)$(somedir)"
+#     so $(MKDIR_P) always has an argument.
+#     We will have better chances of detecting a missing test if
+#     $(MKDIR_P) complains about missing arguments.
+#  3. $(MKDIR_P) is named after `mkdir -p' and we don't expect this
+#     to accept no argument.
+#  4. having something like `mkdir .' in the output is unsightly.
+#
+# On NextStep and OpenStep, the `mkdir' command does not
+# recognize any option.  It will interpret all options as
+# directories to create.
+AN_MAKEVAR([MKDIR_P], [AC_PROG_MKDIR_P])
 AC_DEFUN([AC_PROG_MKDIR_P],
-[AC_REQUIRE([AC_PROG_INSTALL])dnl
+[AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl
+AC_REQUIRE_AUX_FILE([install-sh])dnl
+AC_MSG_CHECKING([for a thread-safe mkdir -p])
 if mkdir -p --version . >/dev/null 2>&1 && test ! -d ./--version; then
-  # We used to define $(mkdir_p) as `mkdir -p .', in order to
-  # allow $(mkdir_p) to be used without argument.  As in
-  #   $(mkdir_p) $(somedir)
-  # where $(somedir) is conditionally defined.  However we don't do
-  # that anymore.
-  #  1. before we restricted the check to GNU mkdir, `mkdir -p .' was
-  #     reported to fail in read-only directories.  The system where this
-  #     happened has been forgotten.
-  #  2. in practice we call $(mkdir_p) on directories such as
-  #       $(mkdir_p) "$(DESTDIR)$(somedir)"
-  #     and we don't want to create $(DESTDIR) if $(somedir) is empty.
-  #     To support the latter case, we have to write
-  #       test -z "$(somedir)" || $(mkdir_p) "$(DESTDIR)$(somedir)"
-  #     so $(mkdir_p) always has an argument.
-  #     We will have better chances of detecting a missing test if
-  #     $(mkdir_p) complains about missing arguments.
-  #  3. $(mkdir_p) is named after `mkdir -p' and we don't expect this
-  #     to accept no argument.
-  #  4. having something like `mkdir .' in the output is unsightly.
-  mkdir_p='mkdir -p'
+  MKDIR_P='mkdir -p'
 else
-  # On NextStep and OpenStep, the `mkdir' command does not
-  # recognize any option.  It will interpret all options as
-  # directories to create.
   for d in ./-p ./--version
   do
     test -d $d && rmdir $d
   done
-  mkdir_p="$INSTALL -d"
+  MKDIR_P="$ac_install_sh -d"
 fi
-AC_SUBST([mkdir_p])
+dnl Do special magic for MKDIR_P instead of AC_SUBST, to get
+dnl relative names right.
+AC_MSG_RESULT([$MKDIR_P])
 ])# AC_PROG_MKDIR_P
 
 
Index: lib/autoconf/status.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/autoconf/status.m4,v
retrieving revision 1.99
diff -p -u -r1.99 status.m4
--- lib/autoconf/status.m4      6 May 2006 05:53:50 -0000       1.99
+++ lib/autoconf/status.m4      13 May 2006 09:51:42 -0000
@@ -493,7 +493,13 @@ AC_PROVIDE_IFELSE([AC_PROG_INSTALL],
   [[\\/$]]* | ?:[[\\/]]* ) ac_INSTALL=$INSTALL ;;
   *) ac_INSTALL=$ac_top_build_prefix$INSTALL ;;
   esac
-])
+])dnl
+AC_PROVIDE_IFELSE([AC_PROG_MKDIR_P],
+[  case $MKDIR_P in
+  [[\\/$]]* | ?:[[\\/]]* ) ac_MKDIR_P=$MKDIR_P ;;
+  *) ac_MKDIR_P=$ac_top_build_prefix$MKDIR_P ;;
+  esac
+])dnl
 _ACEOF
 
 cat >>$CONFIG_STATUS <<\_ACEOF
@@ -538,7 +544,7 @@ dnl During the transition period, this i
 s&@top_builddir@&$ac_top_builddir_sub&;t t[]AC_SUBST_TRACE([top_builddir])
 m4_foreach([_AC_Var], [srcdir, abs_srcdir, top_srcdir, abs_top_srcdir,
                        builddir, abs_builddir,
-                       abs_top_builddir]AC_PROVIDE_IFELSE([AC_PROG_INSTALL], 
[[, INSTALL]]),
+                       abs_top_builddir]AC_PROVIDE_IFELSE([AC_PROG_INSTALL], 
[[, INSTALL]])AC_PROVIDE_IFELSE([AC_PROG_MKDIR_P], [[, MKDIR_P]]),
           [s&@_AC_Var@&$ac_[]_AC_Var&;t t[]AC_SUBST_TRACE(_AC_Var)
 ])dnl
 $ac_datarootdir_hack
@@ -1207,7 +1213,10 @@ ac_pwd='$ac_pwd'
 srcdir='$srcdir'
 AC_PROVIDE_IFELSE([AC_PROG_INSTALL],
 [INSTALL='$INSTALL'
-])
+])dnl
+AC_PROVIDE_IFELSE([AC_PROG_MKDIR_P],
+[MKDIR_P='$MKDIR_P'
+])dnl
 _ACEOF
 
 cat >>$CONFIG_STATUS <<\_ACEOF
Index: lib/m4sugar/m4sh.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/m4sugar/m4sh.m4,v
retrieving revision 1.185
diff -p -u -r1.185 m4sh.m4
--- lib/m4sugar/m4sh.m4 10 May 2006 23:05:16 -0000      1.185
+++ lib/m4sugar/m4sh.m4 13 May 2006 09:51:42 -0000
@@ -918,7 +918,6 @@ m4_define([AS_MKDIR_P],
     done
     test -z "$as_dirs" || eval "mkdir $as_dirs"
   } || test -d "$as_dir" || AS_ERROR([cannot create directory $as_dir]); }dnl
-  # The last "test -d" is in case some other process made the directory.
 ])# AS_MKDIR_P
 
 
Index: tests/local.at
===================================================================
RCS file: /cvsroot/autoconf/autoconf/tests/local.at,v
retrieving revision 1.31
diff -p -u -r1.31 local.at
--- tests/local.at      10 May 2006 23:05:16 -0000      1.31
+++ tests/local.at      13 May 2006 09:51:42 -0000
@@ -292,7 +292,7 @@ if test -f state-env.before && test -f s
       [F77_DUMMY_MAIN|f77_(case|underscore)],
       [FC(_DUMMY_MAIN|FLAGS|LIBS|FLAGS_f)?],
       [ALLOCA|GETLOADAVG_LIBS|KMEM_GROUP|NEED_SETGID|POW_LIB],
-      [AWK|LEX|LEXLIB|LEX_OUTPUT_ROOT|LN_S|M4|mkdir_p|RANLIB|SET_MAKE|YACC],
+      [AWK|LEX|LEXLIB|LEX_OUTPUT_ROOT|LN_S|M4|MKDIR_P|RANLIB|SET_MAKE|YACC],
       [GREP|[EF]GREP|SED],
       address@hidden|.[*#?$].|LINENO|OLDPWD|PIPESTATUS|RANDOM|SECONDS]))=' \
      $act_file 2>/dev/null |