Re: regtests for previous stables failing

[Thomas Morley]

2017-10-26 23:26 GMT+02:00 David Kastrup <address@hidden>:
> Thomas Morley <address@hidden> writes:
>
>> in order to research for
>> https://sourceforge.net/p/testlilyissues/issues/5217/
>> I had the idea to look at regtest behaviour of previous stable lily-versions.
>>
>> Hence I checked out a stable-branch, added a regtest
>> (aaa-voiced-rest-dots.ly) did make, make test-baseline, make check.
>>
>> While it works for current master and the stable/2.20-branch
>> It failed for stable/2.18 and stable/2.16
>>
>> Same behaviour if I _don't_ add a regtest, i.e. make, make
>> test-baseline, make check without any changes.
>>
>> With 2.18 I got a successful make and make test-baseline but make
>> check failed with:
>>
>> ~/lilypond-git/build (dev/2-18-test)$ make check
>> [...]
>> no source for input/regression/out-test/tree.gittxt
>> no source for input/regression/midi/out-test/tree.gittxt
>> 52.137322                      input/regression/aaa-voiced-rest-dots.ly
>> 13.123117                      input/regression/test-output-distance.ly
>>
>>
>> 355 below threshold
>> 3083 unchanged
>> invoking rm -rf /home/hermann/lilypond-git/build/out/test-results
>> mkdir /home/hermann/lilypond-git/build/out/test-results
>> mkdir 
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test
>> mkdir 
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test-baseline
>> input/regression/out-test-baseline/aaa-voiced-rest-dots.ly ->
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test-baseline/aaa-voiced-rest-dots.ly
>> input/regression/out-test/aaa-voiced-rest-dots.ly ->
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test/aaa-voiced-rest-dots.ly
>> invoking gs -sDEVICE=png16m -dGraphicsAlphaBits=4 -dTextAlphaBits=4
>> -slilypond-datadir=input/regression/out-test-baseline/share/lilypond/current
>>   -r101  
>> -sOutputFile=/home/hermann/lilypond-git/build/out/test-results/input/regression/out-test-baseline/aaa-voiced-rest-dots.png
>> -dNOSAFER -dEPSCrop -q -dNOPAUSE
>> input/regression/out-test-baseline/aaa-voiced-rest-dots.eps  -c quit
>> invoking gs -sDEVICE=png16m -dGraphicsAlphaBits=4 -dTextAlphaBits=4
>> -slilypond-datadir=input/regression/out-test/share/lilypond/current
>> -r101  
>> -sOutputFile=/home/hermann/lilypond-git/build/out/test-results/input/regression/out-test/aaa-voiced-rest-dots.png
>> -dNOSAFER -dEPSCrop -q -dNOPAUSE
>> input/regression/out-test/aaa-voiced-rest-dots.eps  -c quit
>> pipe file 
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test-baseline/aaa-voiced-rest-dots.png
>> pipe file 
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test/aaa-voiced-rest-dots.png
>> dir is /tmp/tmpEIBGRh
>> invoking convert -depth 8 -crop 672x414+0+0
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test-baseline/aaa-voiced-rest-dots.png
>> /tmp/tmpEIBGRh/crop1.png
>> invoking convert -depth 8 -crop 672x414+0+0
>> /home/hermann/lilypond-git/build/out/test-results/input/regression/out-test/aaa-voiced-rest-dots.png
>> /tmp/tmpEIBGRh/crop2.png
>> invoking compare -depth 8 -dissimilarity-threshold 1
>> /tmp/tmpEIBGRh/crop1.png /tmp/tmpEIBGRh/crop2.png
>> /tmp/tmpEIBGRh/diff.png
>> Traceback (most recent call last):
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 1342, in <module>
>>     main ()
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 1339, in main
>>     compare_tree_pairs (zip (args[0::2], args[1::2]), out, options.threshold)
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 1060, in compare_tree_pairs
>>     data.create_html_result_page (dest_dir, threshold)
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 1042, in create_html_result_page
>>     link.link_files_for_html (dest_dir)
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 674, in link_files_for_html
>>     compare_png_images (old, new, dest_dir)
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 92, in compare_png_images
>>     system ('compare -depth 8 -dissimilarity-threshold 1
>> %(dir)s/crop1.png %(dir)s/crop2.png %(dir)s/diff.png' % locals ())
>>   File "/home/hermann/lilypond-git/build/scripts/build/out/output-distance",
>> line 1090, in system
>>     assert stat == 0
>> AssertionError
>> rm -rf /tmp/tmpEIBGRh
>> /home/hermann/lilypond-git/GNUmakefile.in:325: recipe for target
>> 'local-check' failed
>> make: *** [local-check] Error 1
>>
>> No idea what wrong.
>>
>> I can provide the full log if needed.
>>
>>
>>
>> With 2.16. even make failed:
>>
>> ~/lilypond-git (dev/2-16-test)$ sh autogen.sh --noconfigure
>> stepmake/aclocal.m4 is newer.  Copying file.
>> stepmake/autogen.sh is newer.  Copying file.
>> processing .
>> Running autoconf ...
>> Skipping configure process.
>>
>>
>> ~/lilypond-git/build (dev/2-16-test)$ make
>> /home/hermann/lilypond-git/stepmake/stepmake/po-targets.make:41: ***
>> recipe commences before first target.  Stop.
>>
>>
>> Any hints to make it work?
>
> make clean
>
> before calling autogen.sh .
>
> Possibly make distclean .

Being at lilypond-git/build (dev/2-18-test) I did:

make clean
make distclean
cd ..
rm -fr build
sh autogen.sh --noconfigure
mkdir -p build/
cd build/
../configure
make -j5
make -j5 test-baseline
make -j5 check


With same failure.

>
> Or just cloning into a fresh directory.

Tomorrow I'll likely try with a fresh clone.

Thanks,
  Harm