Re: Automatically checking regtests (was: Re: Minor releases?)

[Reinhold Kainhofer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am Montag, 17. August 2009 16:08:36 schrieb Michael Käppler:
> [CC to -devel]
>
> > (nobody checks the regression tests for each release, for example
> > -- and that's trivially done with a web browser!)
>
> That reminds me of an idea I recently had: Wouldn't it be possible to
> automatically generate a sort of "checksum" for each regression-test
> output-file and compare it with the former releases? I know that every
> little change in the pdf appearance would lead to a warning then.
> But I'm sure there is a way to do not just "md5sum
> regression-test-foo.pdf" but create a "difference score" by comparing
> the two pdfs.

Isn't this exactly what we already have (make test-baseline to create the 
reference version and then "make test" to compare the new output to the 
reference output...). For the results see 
      http://lilypond.org/test/ 
The 2.13.3 results are at: 
      http://lilypond.org/test/v2.13.3-0/compare-v2-13/index.html)

Graham was referring to the fact that nobody seem to bother about looking at 
those automatically-created regression results before or after a release.

Cheers,
Reinhold
- -- 
- ------------------------------------------------------------------
Reinhold Kainhofer, address@hidden, http://reinhold.kainhofer.com/
 * Financial & Actuarial Math., Vienna Univ. of Technology, Austria
 * http://www.fam.tuwien.ac.at/, DVR: 0005886
 * LilyPond, Music typesetting, http://www.lilypond.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iD8DBQFKiXR1TqjEwhXvPN0RAthBAKCLdUF/UdnNR/hre2Ux9TMrqG9X8QCgqBYK
sbVjSztKwUjlvbNiPG/Tr4w=
=hENM
-----END PGP SIGNATURE-----