Re: critical issues

[Graham Percival]

On Sat, Jan 01, 2011 at 12:31:23AM -0000, Trevor Daniels wrote:
> 
> Graham Percival wrote Friday, December 31, 2010 11:20 PM
> 
> >However, lilypond never intentionally tried to
> >avoid those objects colliding -- in fact, intentionally avoiding
> >this collision would require a fair chunk of extra code.  Should
> >we hold back a stable release just for this?
> 
> Well, I don't intend to die in the ditch over this,
> but the concern I had was this.  Quite a lot of the
> documentation was written, not by inspecting the code
> to see what was intended, but by experimenting and
> writing up what was found.

Yep.

> I certainly worked that
> way, and I think Mark and Keith did recently in
> documenting the new spacing stuff.  In doing that we
> had no idea whether the things we find are intentional
> or a happy coincidence of bugs.

Welcome to lilypond development.  :(

> Now if you can work
> in something about "working as intended or documented"
> wrt determining critical bugs

Nope, for precisely the reason you gave earlier: our documentation
generally has zero input from programmers, so it's not at all a
good representation of "what's intended".

We have a set of "intended to be working" examples.  They're
called the regression tests.  No more, no less.  If a patch breaks
those, then we reject the patch.  If we notice in time.


...

Look, we simply *cannot* offer users anything that would be
"reasonable" by most standards.  We have "highly embarrassing"
bugs from 2006 that we're not even *pretending* to be working on.
We've been in "release crunch" mode for at least six months.  The
only glimmer of hope on the horizon is that, under the most strict
interpretation of "critical regression", almost none of those bugs
were introduced recently.  So there's a chance that once we "catch
up" on the old critical regressions, we won't have many new ones,
and thus we can move forward with a stable foundation.

As a long-time observer and developer, the only honest thing we
can offer users is this:
1. lilypond is distributed without warrantee, including the
implied warantee of fitness for a particular purpose.
(paraphrased from the license)
2. if you find a bug, and make a perfect bug report, then there's
a 90% chance that no programmer will ever look at it.  That chance
increases if you make an imperfect bug report.
3. if you want stability, then never upgrade your lilypond
version.

In my idle moments, I like to discourage myself by trying to
figure out how long it would take to achieve something
"reasonable" for users.  Let's play this game now, and start
making some unrealistic-but-just-possible assumptions:
1. "reasonable" means 100 bugs.  Let's also assume that these can
be the 100 most problematic bugs (i.e. we can fix the easy ones
first without penalizing this "reasonable" goal).
2. with the new lilydev iso, we can gather 10 new programmers.
Keith, Phil, Janek, etc.
3. each of those 10 programmers can fix an average of 1 issue
every 10 days.  By "fix", I mean "understand the problem, read the
code, produce patches for review, respond to comments, make new
drafts, and have somebody push the final fix".
4. the existing developers are capable of doing all the reviewing
for all these patches.
5. we continue to get approximately 1 new issue every 3 days.

I think the above assumptions are unrealistic, but just about
possible.  How long will it take us to reach 100 issues?

Well, we fix 356 issues a year (10 issues per 10 days); let's call
that 350 to make it even.  We add 100 issues a year (1/3*356).  So
our "bug debt" reduces by 250 a year.  We currently have 528
issues, so it'll take us slightly less than 2 years to fix the
easiest 80% of bugs.  Given all those unrealistic assumptions.

Really, things are going to get worse before they get better.  I
fully expect to hit 800 open issues before we manage to start
fixing more issues than we add.

I think our best bet is:
1. stabilize and release 2.14, using the most restrictive
interpretation of "stabilize" and "critical issue" we can.
2. drastically reduce (or abandon entirely) development for 3
months while we sort out the GOP policy questions (many of which
should ease future development)
3. start picking up the pieces and try to recruit more
contributors.

We might achieve "issue count balance" by mid-2011, and if we're
*really* lucky (and/or work hard), we might be able to reduce the
open issue count to less than 500 (and keep it there!) by the end
of 2011.  I know that sounds like a really modest goal, but at the
moment I'm not even optimistic about reaching /that/ target.

- Graham