Re: Question about autocompile bash script

[Jim Long]

On Tue, Sep 10, 2013 at 09:05:49AM -0400, Carl Peterson wrote:
>
> These are along the lines of what I was thinking of. I don't have any
> experience with C. My programming experience as of late is in PHP. So I
> gave some thought last night to using it to read the files and trace the
> \include dependencies. I haven't had a chance to put any code down, but
> I'll probably take a look at that option.

This script is not very robust (it could croak if given malicious
filenames on the \include lines, possibly even causing data
loss), and it has some limitations on the kinds of \include
directives it will recognize (basically, only at the beginning of
a line, with optional leading whitespace), AND it doesn't quite
do what you requested(!), but perhaps it will give you some
ideas.

It takes a master .ly source code filename foo.ly, and a lilypond
output filename (foo.pdf, .eps, .png, etc.), and examines all the
includes in the source tree, and determines whether the output
filename is up to date.  It endeavors to handle recursive
dependencies safely enough to handle circular includes, so that
part of the algorithm is relatively robust, but I'm always open
to suggested improvements.  And there's probably some bugs too,
even apart from the lack of checking on filenames.

I definitely WOULD NOT use this script as-is in production mode.
It's just an instructional example to illustrate a CLI approach
of how to do some of the tasks you're seeking to accomplish.  It
has some security problems that need to be fixed.

This script is written for the bash shell.  Be sure that the
she-bang line reflects the location where bash is installed on
your system.  \include filename paths in the .ly files must
either be absolute, or relative to the present working directory
where the script is run.

HTH!

Jim


Given these timestamps:

$ ls -lt top.ly [A-L].ly top.png
-rw-------  1 james  wheel  16 Sep 12 00:07 F.ly
-rw-------  1 james  wheel   0 Sep 12 00:07 top.png
-rw-------  1 james  wheel  48 Sep 11 23:55 top.ly
-rw-------  1 james  wheel  16 Sep 11 23:52 L.ly
-rw-------  1 james  wheel  52 Sep 11 23:28 A.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 D.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 E.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 G.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 H.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 I.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 J.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 K.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 B.ly
-rw-------  1 james  wheel  16 Sep 11 23:17 C.ly

The script gives this output:

$ ./include-tree.sh top.ly top.png
file "top.ly" includes: "A.ly" "F.ly" "J.ly"
file "A.ly" includes: "B.ly"
file "F.ly" includes: "G.ly"
file "J.ly" includes: "K.ly"
file "B.ly" includes: "C.ly"
file "G.ly" includes: "H.ly"
file "K.ly" includes: "L.ly"
file "C.ly" includes: "D.ly"
file "H.ly" includes: "I.ly"
file "L.ly" includes: "A.ly"
file "D.ly" includes: "E.ly"
file "I.ly" includes: "J.ly"
File "A.ly" has already been read
file "E.ly" includes: "F.ly"
File "J.ly" has already been read
File "F.ly" has already been read
top.png is not up to date




#!/usr/local/bin/bash

# Script to accept a lilypond source filename and a lilypond
# output filename.  The source filename is scanned for
# includes, and all the includes are scanned, ad infinitum.

# Once all the individual source filenames are known, the 
# timestamps are compared to the timestamp of the output
# filename.  If the output filename is the newest of all
# the files, we return success, else we return failure.

# Some restrictions apply to the contexts in which the
# lilypond \include directive is recognized:

#   - it must begin a line, or have only whitespace before it
#   - \include'd filenames must not contain unreasonable characters,
#     and no checking is done to ensure compliance.  Data loss could result.


includes_found_in_file()
{

# pass $1 with a filename, including quotes
# we'll output a list of all the \include'd filenames
# found in $1, one to a line, with quotes around them.

# at this time, we only recognize \include's that are on a
# line by themselves, with optional leading whitespace.
# \include'd filenames may not contain whitespace characters.

        eval "egrep '^[[:blank:]]*[\]include[[:blank:]]*\"[^[:blank:]\"]*\"' 
$1" | \
        awk {'print $2'} 

} # includes_found_in_file


already_read()

# pass $1 with a single filename
# pass $2 with a space-delimited list of files already read
# we'll return true iff $1 is in the list $2

{
        egrep -qF " $1 " <<< "$2"
} # already_read


############################
#
# main routine 
#
############################

#  pass $1 with the name of the top-level .ly file
#  pass $2 with the name of lilypond output file (.pdf/.png/.eps, whatever)

#  we'll return success iff the output file is newer than
#  all of the source files, else we'll return failure

#  Put quotes around the top-level filename, assuming it exists

[ -f "$1" ] && ly_file=\"$1\" || {
        echo "File $1 not found"
        exit 2
}

target_file="$2"

if [ ! -f "$target_file" ]
then
#       echo $target_file not found
        echo $target_file is not up to date
        exit 1
fi

shift 2

#  First, we find all the files that are included by the top-level
#  .ly file, as well any nested includes.

files_read=" "  # keep track of which files we've read through so far

set $ly_file    # start off reading the top-level file

finished=false

until $finished
do

        filename="$1"

# have we already read this file?

        if already_read "$filename" "$files_read"
        then
                echo File "$filename" has already been read
        else

#               echo Reading "$filename" ...

# find all the '\include' references within this one file
                includes=$(includes_found_in_file "$filename")
                echo file "$filename" includes: $includes

# now that we've read this file, concat the filename onto files_read
                files_read="$files_read $filename"
# and concat the included names found to the files we need to read
                set "$@" $includes

        fi

        shift   # done with this file, move on to the next

        [ -z "$1" ] && finished=true    # terminate when we run out of names

done

#echo all told, the included files are:
#echo $files_read

newest_file=$(eval ls -t $files_read $target_file | head -1)

#echo The newest file is $newest_file

if [ "$newest_file" = "$target_file" ]
then
        echo $target_file is up to date
        exit 0
else
        echo $target_file is not up to date
        exit 1
fi