Re: regression volunteer

[Graham Percival]

Alexander Deubelbeiss wrote:
> -------- Original-Nachricht --------
> > Datum: Thu, 29 Nov 2007 15:49:32 +0100 Von: Mats Bengtsson
> > <address@hidden>
>
> > A much more clever way is to use the automated regression testing
> > system,
>
> Extremely clever for "day to day" type checks, but for a first pass
> the actual, individual tests and results need some love. Take a look,
> for instance, at accidental-single-double.ly: This works, and
> probably shows extremely little variation from release to release,
> but it doesn't test what the description says it's meant to test. The
> notes are "gisis gis gisis ges"; according to the text they should be
> "gisis gis geses ges".
>
> So, yeah. I'm makin' a list and checkin' it twice ;)

Stan Sanderson also expressed interest, so let's have two people making 
lists and checking them twice.  :)

Here's the instructions I sent to Stan:


Unstable docs, developer resources:
http://lilypond.org/doc/v2.11/Documentation/devel


Take a look at the regression tests.  WARNING: I think it's about 5 
megabytes.
http://lilypond.org/doc/v2.11/input/regression/collated-files

If any example looks broken, send an email to bug-lilypond ASAP.  You 
just need to state the test name.  (although one or two sentences about 
the problem could be nice -- depends on how obvious the problem is)


A better method, once you've checked the complete list of tests, is the 
comparisons between releases.  For example:
http://lilypond.org/test/v2.11.33-1/compare-v2.11.32-1/index.html

These are the webpages you will be checking most often.  They show you 
the examples that changed.  Most changes will be harmless -- for 
example, the changes on that webpage are fine.  But some changes will 
break things; if the picture on the right-hand column looks bad, send us 
the file name.


Start off looking at these webpages, and ask any more questions.  I'd 
like you to check these regtests (just the comparison) every time 
there's a new devel release.  This averages two per month, although in 
the past two months there has only been one update.  So the amount of 
time will vary somewhat... but as you can see, it's not such a huge job.


INSTRUCTION FOR BOTH:
- please sign up for the bug-lilypond mailist.
- when you find a problem (be it in the release-to-release comparison, 
or in the complete list), send an email to bug-lilypond.
- if the other person has already reported a bug, don't send another 
email.  :)

Cheers,
- Graham