Fix for some kinds of "strange dependency behavior"

[tom surace]

I have recently solved a problem where dependents don't seem to be 
compiled when using Command. I am relieved to find that dependencies are 
not broken!

It turns out that the real problem was that the commands were run on a 
remote machine via ssh, mounted over an NFS partition. The files didn't 
exist yet when cons went to generate a new signature. Because of the way 
sig::signature "works", this results in a signature of ''. Thus anything 
depending on it is never updated. Problem solved.

If you never have targets that depend on nonexistant files, you can 
modify signature so that it never returns '', and easily avoid this 
difficult-to-debug problem. I enclose a signature program that should be 
identical to the builtin except for the lack of the "null signature 
generation" feature.

Another approach would be to "stat" targets after they are built, as a 
sanity check. This is left as an exercise to the cons hacker.

Hope this helps somebody. (It has now helped me...twice.)

#
# sig::signature -- rewritten signature file dies if the
#   file does not exist. This prevents the build from quietly
#   failing to update dependencies when NFS-mounted directories
#   have not been updated, and other unexpected conditions.
#
# Determine the current signature of an already-existing or
# non-existant file.  Unless a specific signature type (bsig
# or csig) is requested, this consults the file's signature
# array to decide whether to return content or build signature,
# and whether to use a cached value from a .consign file.
sub sig::signature {
    my($self, $file, $sigtype) = @_;
    $sigtype = $file->sigtype if ! $sigtype;

    # Anything except 'build' or 'content' is a typo
    die( "unknown SIGNATURE type for file'" . $file->path . "'" ) if not 
$sigtype;       

    #open(TTY, ">/dev/tty");
    #print TTY $file->path, ": $sigtype\n";
    #close(TTY);
    my($path) = $file->path;
    my $sig = '';               # return value

    # Note: this code allows up to 1 minute of retries before failing,
    #       which seems to be ok for 98% of builds over a smb share.
    #
    # If it's on a remote filesystem, let's give it two seconds
    # to appear (which is to say, three tries). This will not solve
    # NFS problems, which can take a minute or two to resolve, but
    # may work around occasional win32 SMB filesystem problems.
    my( $time );                # file timestamp
    my( $Tries ) = 0;
    TRY: while( 1 )             # loop
    {
      $time = (stat($path))[9];
      if( $time )
      {
        last TRY;               # yay
      }
      elsif( $Tries < 60 )   # keep trying?
      {
        warn( $file->path . " does not exist, waiting. (${Tries})\n" );
        sleep 1;
        $Tries++;               # here comes another one
      }
      else                      # give up
      {
        # I have decided to disallow returning '' for nonexistant files.
        # If you don't want a sig, don't call "signature".
        die( "sig::signature could not stat file '" . $path
	     . "', if it was generated by an external command, be sure it exists 
before returning." );
        
        # what the code used to do instead of dying
        # $file->{$sigtype} = '';    
        # $sig = '';
      }
    }

    if ($file->{$sigtype})
    {
      $sig = $file->{$sigtype};
    }
    elsif ($file->is_on_rpath || $file->stored)
    {
      # Fetch the cached sig? I think.
      if ('sig'->fetch($file) && $file->{$sigtype})
      {
        if ($file->{'sigtime'} == $time || # if timestamp unchanged
            ! $param::rep_sig_times_ok     # or not using timestamps
            && $file->is_on_rpath)         # and is in repository
        {
          $sig = $file->{$sigtype}; # use cached sig. I think.
        }
        else
        {
          $file->{$sigtype} = undef;
        }
      }
      else
      {
        $file->{$sigtype} = undef;
      }
    }

    # If we're getting the file from a repository and the filename is
    # not absolute, uh, call bsig or csig to generate a signature,
    # I think. Is it just me or does this function do a lot of
    # magic things?

    if( !$sig )                 # didn't get it yet?
    {
      if ($file->is_on_rpath || ! File::Spec->file_name_is_absolute($path))
      {
        if ($sigtype eq 'bsig')
        {
          $sig = $self->bsig($file);
        }
        elsif ($sigtype eq 'csig')
        {
          $sig = $self->csig($file);
        }
        else
        {
          # t/DD: not having a sig for a file can be the source of much
          #      frustration. I think there's a default anyway. Let's not
          #      continue here.
          die( "Unexpected sigtype defined for file '" . $file->path . "': '"
	       . $sigtype . "', fix your SIGNATURE environment for this file (see 
Cons documentation)" );
        }
      }
      else
      {
        # This file is not in a repository or under the local directory
        # structure.  In the canonical case, it's a utility that will be
        # executed by a command.  Historically, Cons has returned the
        # name of the command concatenated with the modification time.
        # Note that this is *not* the path ("cc" not "/bin/cc"), so it
        # would lose in the unlikely event that a different copy of the
        # utility was used that happened to have the same modification
        # time (due to living in a different directory on the PATH, for
        # example).  The obvious "fix" of using the path like so, however:
        #       return $path . $time;
        # is wrong.  In a multi-machine build environment, different
        # systems may have the same utility in different locations (due
        # to different NFS mount points, for example), which would
        # cause a lot of unnecessary builds if we used the full path.
        # A better solution to strengthen this signature would be to
        # also concatenate the size of the file, but that would cause
        # unnecessary rebuilds when coming from .consign files that used
        # the old scheme.  All of which is to merely explain why we're
        # leaving this as it has been, but documenting it here in case
        # there's reason to change it in the future.
        #
        # t/DD: This is less than useless if you run remote commands
        #       via ssh or "cd && command", because then you just get the
        #       name/timestamp for "cd" or "ssh" here. :I Oh well.
        $sig = $file->{entry} . $time;
      }
    }

    $sig || die( "Null signature calculated by accident, fix program 
flow in sub sig::signature" );

    return $sig;
}


t/DD
--