Re: testing out Docker CI scripts?

[David Kastrup]

Jean-Charles Malahieude <address@hidden> writes:

> Le 23/02/2020 à 15:56, Han-Wen Nienhuys a écrit :
>> On Sun, Feb 23, 2020 at 11:44 AM Han-Wen Nienhuys <address@hidden> wrote:
>>> and initially it is working as expected (4 cpus.)
>>>
>>> 11:12:40 3.23  Making Documentation/out-www/music-glossary.pdf < texi
>>> 11:12:41 3.30  Making Documentation/out-www/notation.pdf < texi
>>> 11:12:47 3.43  Making Documentation/out-www/snippets.pdf < texi
>>>
>>> but later in the process, I get
>>>
>>> 11:17:54 2.07  Making Documentation/de/out-www/notation.pdf < texi
>>> ..
>>> 11:18:46 1.85  Making Documentation/es/out-www/notation.texi < tely
>>> 11:19:25 2.51  Making Documentation/es/out-www/learning.texi < tely
>>>
>>> ie. the build process for other languages is leaving ~2 CPUs idle, for
>>> about 15 minutes. This means it should be doable to shave off 7.5
>>> minutes off the make doc build.
>> I should have taken more detailed notes on the first runtime.
>> I tried some more work with parallelization, which yields an overall
>> runtime of 36m, which shows that I haven't been able to disprove
>> David. It's a ~5 min improvement. Probably we should look more deeply
>> into making lp-book not be exclusive.
>> 15:17:11 2.73  Making Documentation/de/out-www/learning.pdf < texi
>> 15:17:17 2.91  Making Documentation/de/out-www/notation.pdf < texi
>> ..
>> 15:18:13 2.26  Making Documentation/es/out-www/learning.texi < tely
>> 15:18:14 2.26  Making Documentation/es/out-www/notation.texi < tely
>> real 36m12.256s
>> user 79m24.432s
>> sys 10m36.464s
>> For posterity, during the lp-book processing for doc/web, lilypond
>> is
>> busy at about 40%. The rest presumably goes to ghostscript and image
>> scaling.
>> Compared to just the regtests, that is 2.5x more files and 2.5x more
>> expensive processing = 6x more expensive overall. Since the regtest
>> completes in about 3 minutes, the equivalent for "make doc" would be
>> 18 mins. So we still are missing a factor two. It looks like the poor
>> parallelism of the build process can explain that remaining factor.
>> 
>
>
> FWIW, on my Fedora31 box, Intel® Core™ i5-4440 CPU @ 3.10GHz with 8
> Gigx of RAM, on the translation branch as of 
> 5149cf967a8b9f5c01696a3bd994c5ca6f2fa473 (Fix some dead links in
> German documentation):
>
> -*- mode: compilation; default-directory: "~/GIT/Traduc/" -*-
> Compilation started at Sun Feb 23 12:14:23
>
> ./autogen.sh && make -j5 && make -j5 doc
> […]
> make[1]: Leaving directory '/home/jcharles/GIT/Traduc'
>
> Compilation finished at Sun Feb 23 12:46:46

Why wouldn't either of you be using the CPU_COUNT= ... environment
variable for letting lilypond-book run parallel jobs of lilypond ?

-- 
David Kastrup
My replies have a tendency to cause friction.  To help mitigating
damage, feel free to forward problematic posts to me adding a subject
like "timeout 1d" (for a suggested timeout of 1 day) or "offensive".