lilypond-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: `make check' overworks one core on my Core2 quad


From: Han-Wen Nienhuys
Subject: Re: `make check' overworks one core on my Core2 quad
Date: Sun, 13 Dec 2009 13:55:39 -0200

Oh wait - there is one thing I did not think about: snippets may be
shared by different documents, so if you use make -jX it is
conceivable that make invokes two separate lilypond processes that
have non-empty intersection of their arguments.

For now, the easy fix is to use -j1 with CPU_COUNT for building the docs.

A more elaborate solution would be either some kind of locking, or to
check whether the .ps / .pdf exists before actually processing the
.ly; the latter is still suscepitible to races, though, but a check
could make the opportunity window smaller.

A last solution would be to unshare the snippet database, but that
would mean a lot of double processing and extra disk usage, because
every translation would have to do 90% of the snippets all over again,
and store them separately.

On Sun, Dec 13, 2009 at 1:48 PM, David Kastrup <address@hidden> wrote:
> Han-Wen Nienhuys <address@hidden> writes:
>


>>> If we are talking about 2 particular files colliding.  If we are talking
>>> about a collision in n files, there are n(n+1)/2 combinations all of
>>> which have a 2^20 chance of collision (of course, those are not
>>> independent collisions, but the approximation is pretty good).  A 1%
>>> chance of collision is (first order approximation) achieved when
>>> n(n+1)/2=10000, meaning n is something like 140.
>>>
>>> Close enough to make 40 bits feel uncomfortable.
>>
>> This is trivially fixed by changing the line
>>
>>             ## let's not create too long names.
>>             self.checksum = hash.hexdigest ()[:10]
>>
>> in lilypond-book.py, I doubt that this will fix the problem though.
>>
>> Note that the proposed race condition would also require the colliding
>> files to be processed by different lilypond subprocesses, and be
>> passing through the (ps -> pdf) conversion of the same file at the
>> same time.
>>
>> theoretically, not impossible but I would not bet my money on this
>> being the problem.
>
> Well, make does the equivalent of a tsort on its dependencies, and I
> would not surprised if the tiebreaker was an alphabetical sort on
> filenames.
>
> So if the hashed filenames pass through make (no idea if they do), even
> just two threads are likely to be be hit by a hash collision.

-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen




reply via email to

[Prev in Thread] Current Thread [Next in Thread]