Re: How can I get rid of dinkleberry autotools temp dirs?

[Ralf Wildenhues]

Hi Bruce,

Sorry for a late reply.

* Bruce Korb wrote on Sun, Oct 30, 2005 at 05:51:43PM CET:
> 
> Attached is a log of a couple of runs of autoreconf and the
> resulting consequences.  Things of note:
> 
> 1.  with the ``--debug'' option, autoreconf exits with status "0"
>     but without it, "1".  Bad.

Hmm.  As documented,
| `--debug'
| `-d'
|      Don't remove the temporary files.

using --debug will keep the files on purpose.  Without it, it fails,
exactly *because* the error happens in the cleanup process:
> >     autoreconf: Leaving directory `.'
> >     autoreconf: cannot empty /local/tmp/ar8810.11683: Is a directory

Not nice, but this sort of Heisenbug can't be avoided completely with
the given semantics.  Of course there is a couple of real bugs hidden,
namely that the error message is misleading, and that
/local/tmp/ar8810.11683 *should* be empty in this case, i.e. autom4te
and autoheader should have removed these:

> >     /local/tmp/ar8810.11683/am4tNcVwla
> >     /local/tmp/ar8810.11683/am4t0eEdr4
> >     /local/tmp/ar8810.11683/am4tVDkyK5
> >     /local/tmp/ar8810.11683/ahOnypvr
> >     /local/tmp/ar8810.11683/am4tfAHR8o
> >     /local/tmp/ar8810.11683/am4te7Ws2L

BTW, it took me quite a while to realize the prefix of the lowest
directories was `am4t' as in `autom4te', and automake was not involved..

First proposed patch: improve the error message of the removal failure
(see below for more text).  OK to apply?

        * lib/Autom4te/General.pm (END): Show which entry in the `$tmp'
        directory caused the unlink failure.

Index: lib/Autom4te/General.pm
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/Autom4te/General.pm,v
retrieving revision 1.34
diff -u -r1.34 General.pm
--- lib/Autom4te/General.pm     14 May 2005 07:00:39 -0000      1.34
+++ lib/Autom4te/General.pm     11 Jan 2006 13:54:40 -0000
@@ -193,11 +193,14 @@
     {
       if (<$tmp/*>)
        {
-         if (! unlink <$tmp/*>)
+         while (<$tmp/*)
            {
-             print STDERR "$me: cannot empty $tmp: $!\n";
-             $? = 1;
-             return;
+             if (! unlink $_)
+               {
+                 print STDERR "$me: cannot empty $tmp ($_): $!\n";
+                 $? = 1;
+                 return;
+               }
            }
        }
       if (! rmdir $tmp)

(end of patch)

> 2.  All the perl autotools tools use a function "mktmpdir" which
>     seems to have some magical properties.  This function is not
>     documented in any of the Perl man pages I have access to.  It
>     seems that it is expected that the directory will just disappear
>     at program termination.  It does not.

See autoconf/lib/Autom4te/General.pm for the implementation and
documentation.  It's internal to Autoconf.  Without --debug, it should
just disappear at program termination: the END section in that file has
provisions for this.

> 3.  Autoreconf sets TMPDIR to its temporary directory so that spawned
>     tasks (autom4te et al.) use that new directory for their temp
>     home.

Yes.  I think this is a good idea!

> 4.  Because these tools wind up leaving their temporary directories
>     around, you get the error message:

ACK.

> 5.  I modified autom4te and autoheader to have this line of text at the end:
>          xsystem ("set -x ; rm -rf $tmp");
>     but you notice that some "am4t*" and "ah*" directories persist.
> 
> 6.  Adding that to autoreconf fixes the problem, until I get the
>     next edition of autoconf.  :)

Let's fix the bug(s) in autom4te/autoheader/whereever that causes it to
leave directories around in your case, so you don't have to forward-port
your workaround.

* Bruce Korb wrote on Sun, Oct 30, 2005 at 05:59:32PM CET:
> 
> Even doing that left a directory:
> 
> > $ find /local/tmp/a*
> > /local/tmp/arRdqYXc
*snip*

> so now I know the problem:
> 
> mktmpdir is creating *TWO* directories.  e.g. /local/tmp/arRdqYXc and
> /local/tmp/ar8810.11683 and $tmp is set to the latter.  This second
> directory is properly removed (if it is empty) at the end of the Perl
> run.  The former is not.  If these autoheader/autom4te "former" temp
> directories are left in the autoreconf temp directory, you have a
> problem.
> 
> So, the question boils down to how come two directories are created
> and what can I do about it?  :)

Hmm.  I also see that in some cases temp files are left over sometime,
could not reproduce a case yet though.

Could you please step through mktmpdir with the perl debugger to make
sure that it really is creating 2 directories per invocation?  With the
patch below you can find out whether mktemp(1) may have issues, e.g.
return an error but still create the directory.  This seems to me the
most likely explanation for your issue.  (Or leave in the `-q' -- maybe
that's what upsets mktemp?)

Which system are you on?  Where is mktemp(1) from?

Cheers,
Ralf

Index: lib/Autom4te/General.pm
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/Autom4te/General.pm,v
retrieving revision 1.34
diff -u -r1.34 General.pm
--- lib/Autom4te/General.pm     14 May 2005 07:00:39 -0000      1.34
+++ lib/Autom4te/General.pm     11 Jan 2006 13:52:21 -0000
@@ -312,7 +312,7 @@
 
   # If mktemp supports dirs, use it.
   $tmp = `(umask 077 &&
-          mktemp -d -q "$TMPDIR/${signature}XXXXXX") 2>/dev/null`;
+          mktemp -d "$TMPDIR/${signature}XXXXXX")`;
   chomp $tmp;
 
   if (!$tmp || ! -d $tmp)