Re: GOP-PROP 4: lessons from 2.14

[Graham Percival]

On Thu, Jun 30, 2011 at 11:17:36AM -0300, Han-Wen Nienhuys wrote:
> On Wed, Jun 29, 2011 at 7:26 PM, Graham Percival
> <address@hidden> wrote:
> > http://lilypond.org/~graham/gop/gop_4.html
> >
> > What went well, what went badly? This is a discussion only; it
> > will be summarized, and we will refer back to it in future policy
> > decisions, but no new policies will be decided in this round.
> 
> Overall, I think this cycle took too long.

Agreed.

> We should strive to have policies that make each development release
> be a worthy stable candidate. That means -for example-

-snip suggesions-

I'd like to keep this particular discussion focused on the history
and our interpretation/opinion of it.  There will probably be some
obvious policy proposals that we can make from that discussion,
but I'd rather avoid proposals entirely at this stage.

> We should examine other reasons for the delay of the 1st candidate
> appearing, and the delay between the 1st and last release candidate,
> and figure out if there is a way to prevent them from causing large
> delays again.

Nice way to split it up!

Regular 2.13 releases did not happen until 2009-10; it was
therefore impossible to have a release candidate before then.
Between 2009-10 and 2011-01 (first release candidate), my vague
sense is that roughly 50% of Critical bugs originated from before
2010-08 (when Phil began checking regtests).
It would be nice if somebody checked into this... maybe select 10%
of the bugs, then look at when they were introduced?  (most
regression bug discussions began by finding the first version that
broke)

My vague sense of 2011-01 to 2011-06 was that about 20% of
Critical regressions were introduced before 2010-08.  Release
candidate 4 or 5 (can't remember which) was almost good enough,
but the merging of the beam collisions and midi rewrite introduced
5-10 Critical bugs, pushing the release into June.  Also, my
absence in May (vacation) added a 2-3 week delay in releases.
(similarly, it would be nice if somebody checked into these
recollections)

I will note that many of the Critical bugs from the beams and midi
came from use cases that we did not anticipate (for midi:
something about quicktime on windows?), and had no regression
tests for.


> On a tangent: at Google I am working on a side-project that
> essentially is distcc on steroids; it will allow any compilation
...
> It could speed up GUB and regtest checking for those that have access
> to several machines.

Cool project, but I don't think our limitation is processing
power.  My desktop is idle at least 95% of the time, so I could
dedicate a lot more horsepower to GUB if necessary, and the
problem with regtest checking is one of organization: managing
volunteers to look at the output.

Daily tests would be awesome:
http://code.google.com/p/lilypond/issues/detail?id=933
but it would take 1-2 hours to write the scripts to update git,
compile lilypond, and send emails if stuff broke.  I'm not likely
to have that amount of time to spend on a "pet project" until Sep
or Oct.

Cheers,
- Graham