[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

merging regtests

From: Graham Percival
Subject: merging regtests
Date: Tue, 30 Aug 2011 06:25:44 +0100
User-agent: Mutt/1.5.20 (2009-06-14)

We haven't had a release in a month.  More to the point, we
haven't had a regtest comparison in a month.  But we've had lots
of patches go in.

I will bet money that we have at least 3 regressions.  I will also
bet money that at least 1 of them is from a developer who is going
on vacation soon.  (not because he's bad, but just because he
produces a lot of patches that touch a lot of code)
That means that we'll be back to the wonderful pre-2.14 days when
helpful programmers are struggling to fix problems in other
people's code.
is not going away any time soon.  It involves complicated stuff in
the build system, which nobody wants to touch.

As a temporary workaround, I will now reduce the number of
regression tests.  In particular, I will merge the segfault
regtests -- we don't care about the output of those; we just want
to make sure that lilypond doesn't crash.  This will hopefully be
sufficient; the build only broke a month ago, so eliminating 5 or
so filenames should be enough.

If that is not sufficient, I will look at merging or even deleting
other regtests.

In the long term, we want to have a discussion about organizing
the regtests better, quite possibly into separate directories.  In
the medium term, we want to have a build system which doesn't
limit the number of ly files in a directory.

In the short term though, we **need** regular development
releases.  Our review process is not perfect, the countdowns
aren't perfect, the amount of testing that developers do before
pushing patches isn't perfect.  The only feasible way of
stabilizing the process is to have regtest comparisons and get
users to test development releases.

- Graham

reply via email to

[Prev in Thread] Current Thread [Next in Thread]