Re: Policy about not-yet-regression tests?

[Graham Percival]

On Sat, Apr 09, 2011 at 07:28:00AM +0200, David Kastrup wrote:
> How do I notice that somebody fixed the bug by accident if I am not
> allowed to add a check for the bug to the regtests?
> 
> Am I supposed to keep a _private_ regtest for bugs I report, checking it
> periodically to make sure that the bug is still there?
> 
> That makes me a single point of failure.  I am not confident that this
> is a good idea...

I'm not confident that it's a good idea either, but I'm not
confident that I really understand the idea at all.

You are concerned that we accidentally fix bugs so often that you
would like an automatic method of checking for those bug fixes?

... well, I suppose that it _does_ happen, and it could be useful
to have such a system.  But remember that we only consider
something to be a Critical regression if it deliberately worked
before -- if a bug goes away by accdient, then it reappears, we
don't consider that to be Critical.  That said, it could be useful
to know exactly when it disappeared/reappeared.

...

At the moment, though, I think such an effort should remain
outside of the main code base.  It doesn't necessarily need to be
private; you could add a github project or something.  But unless
you have some good evidence that this happens much more often than
I suspect, I think that adding such a system to the main lilypond
build would be more trouble than it was worth.  (also remember
that the bug squad members have no git push ability, nor the
technical knowledge to use it even if they had it)

Cheers,
- Graham