Re: Macro for(UP_and_DOWN) and 3 similar. (issue 2491) (issue 6109046)

[David Kastrup]

Łukasz Czerwiński <address@hidden> writes:

> I have never ever get an email from Google Code. I have just checked
> that triple. That's the reason for ignoring your comments. I'm sorry
> that my new patch made you run your tests twice to give me the same
> list of errors...
>  
> Do you run tests for each patch uploaded to Rietveld?

I won't any more.  It is high time somebody else pitches in.  Neither my
computing power nor my personality make me suitable for that job.

>     > I'd like to know how to run regtests. Should I
>     > follow:
>     http://lilypond.org/doc/v2.15/Documentation/contributor/regtest-comparison
>     >
>     > and compare all those tests that differ?
>     
>     
>     Yes, that is the respective instruction.  You get a visual
>     comparison
>     for the tests that differ significantly.
>     
>
> Ok, thank you. But compiling and running regtests took approx. 2.5
> hours. It's very long...

Half of that is setting the baseline.  It should be somewhat less: I
think it took about 2 hours on my previous laptop (10 years old).  When
it broke down, another senior developer sent me the money for a model
that is 5 years old (Intel Core 2 Duo at 1.8GHz).  A full comparison
including baseline is now something short of 40 minutes (if there are
several patches in the queue, only a single baseline gets made).  Pretty
much every other developer has more powerful hardware available: some
even has been explicitly bought for LilyPond work, and the fully
automated tests for pushing to master, quite more expensive than the
patch testing, are run on such a system.

The patch testing, however, requires humans to actually examine the test
results.

> Now I have them, but don't know, how to read the details.

Neither do I.

> Could you give me some tips what those numbers in HTML from the
> attachment are?  Probably some distances, but why there are 7 times:
> Stem: 1.00000 and then Barline, while there are only 4 notes before
> the first barline?

Most stuff of interest is the visual comparison.  The numbers are only
interesting if you _know_ that there should be no difference whatsoever,
and you can't figure out visually what changed.  The logs are important
if warnings or similar stuff changes.  And so on.

There is one always-changing regtest designed to change, and there are a
few regtest outputs that tend to change for no discernible reason (like
the graphviz output).  Hopefully we figure them out one day.

But sometimes there is a _lot_ of change when something goes wrong.
Ignoring that is not a good idea.

-- 
David Kastrup