Re: race condition in writing lilypond-book filenames

[John Mandereau]

Le dimanche 13 décembre 2009 à 15:55 +0000, Graham Percival a écrit :
> diff --git a/scripts/lilypond-book.py b/scripts/lilypond-book.py
> index 392ddd0..b9731f1 100644
> --- a/scripts/lilypond-book.py
> +++ b/scripts/lilypond-book.py
> @@ -1273,7 +1273,11 @@ left-margin-default right-margin-default)"
>          directory = os.path.split(path)[0]
>          if not os.path.isdir (directory):
>              os.makedirs (directory)
> -        out = file (path + '.ly', 'w')
> +        filename = path + '.ly'
> +        if os.path.exists(filename):
> +            print "PANIC!  ", filename, "already exists!"
> +            sys.exit(1)
> +        out = file (filename, 'w')

It's much more informative to print a diff against exiting file to see
what goes wrong:

diff --git a/scripts/lilypond-book.py b/scripts/lilypond-book.py
index 392ddd0..f03b9cb 100644
--- a/scripts/lilypond-book.py
+++ b/scripts/lilypond-book.py
@@ -1264,7 +1264,7 @@ left-margin-default right-margin-default)"
 
     def basename (self):
         cs = self.get_checksum ()
-        name = '%s/lily-%s' % (cs[:2], cs[2:10])
+        name = '%s/lily-%s' % (cs[:2], cs[2:])
         return name
 
     def write_ly (self):
@@ -1273,7 +1273,11 @@ left-margin-default right-margin-default)"
         directory = os.path.split(path)[0]
         if not os.path.isdir (directory):
             os.makedirs (directory)
-        out = file (path + '.ly', 'w')
+        filename = path + '.ly'
+        if os.path.exists (filename):
+            warning ("%s: duplicate filename, printing diff against existing 
file" % filename)
+            ly.stderr_write (filter_pipe (self.full_ly (), 'diff -us %s -' % 
filename))
+        out = file (filename, 'w')
         out.write (self.full_ly ())
         file (path + '.txt', 'w').write ('image of music')
 

> I'm honestly surprised I got this with a simple "make" instead of "make doc".
> ... oh wait, this file is <c e g>, which is at the beginning of the
> LM.  Of course it would generate the same file multiple times... once
> for the HTML output, once for PDF, maybe a third time for info?
> ... oh wait *2, just running "make" shouldn't generate the HTML/PDF.
> Hmm... maybe the second time through, it was generating a translation
> of the LM?

"make all" generates Info docs without images, and for this purpose
lilypond-book runs but calls "true" instead of "lilypond", in order not
to generate images of music.


> In any case, we definitely have the same filename being used multiple
> times when building lilypond.  Is this a problem?  I can't say for
> certainty, but I *do* know that unpredictable race conditions occur in
> the docs when building with more than one process.

With the patch above applied, I did

rm -rf out/lybook-db
make -C Documentation clean
make -C Documentation &> duplicate_snippets_diffs.log

I attached relevant parts of duplicate_snippets_diffs.log, which show
that lilypond-book.py:relevant_contents() may forget a bit too many
fragment options.  I'll work later about this issue, hopefully before
next week-end.

Best,
John

duplicate_snippets_diffs.log.bz2
Description: application/bzip

signature.asc
Description: Ceci est une partie de message numériquement signée