quality assurance (was: Re: GNUstep Base 1.11.0)

[Lars Sonchocky-Helldorf]

Am Samstag, 23.07.05 um 13:47 Uhr schrieb Benoit Astruc:

> Hello,
>
>
> I just tried to compil GNUstep Base 1.11.0, freshly download from  
> GNUstep http) and I found that there is an error GSXML.m at line 5 142  
> :
>
> request = [tequest initWithURL: [NSURL URLWithString: connectionURL]];
>
> should be
>
> request = [request initWithURL: [NSURL URLWithString: connectionURL]];
>
> I suppose.
>
> Maybe it's corrected in CVS but it seems quite bad to have such error  
> in the last stable version.

IIRC the odd minor release number (11 in 1.11.0) points out that this  
is an unstable release of GNUstep.


But nevertheless I think it is a bad sign that this kind of a bug  
(doesn't compile) could creep into a *release* of GNUstep. And even  
worse, it would not have been detected if Benoit would not have built  
this special configuration. The bug has gone unnoticed for a long time,  
it has been introduced with:

http://savannah.gnu.org/cgi-bin/viewcvs/gnustep/gnustep/core/base/ 
Source/Additions/GSXML.m.diff?r1=1.78&r2=1.79

on Sat Mar 12 17:38:18 2005 annotated with "Patch to port GSXMLRPC to  
MacOSX" which tells me the submitter did *not* test his patch before  
committing. Otherwise he would have experienced what Benoit experienced.


Please don't get me wrong here. I don't want to do finger pointing  
because of a small bug that only occurs in border cases. My issue is  
something else:

While we keep fixing things here while inadvertently destroying things  
there.

This shows that we could need some sort of quality assurance in our  
development process. Otherwise GNUstep will never become really mature.


There are several things we could do in this regard (all stolen from  
the GCC team)

CVS check-ins:

- require at least a compile test before things get checked in (this  
would catch this sort of bugs)

- peer review of patches to be checked in

- do a regression test before checking in things (needs test cases, see  
below)

bug fixes:

- write test cases that prove a bug is fixed

releases:

- create a CVS branch (not just a tag) for each release (this helps to  
insure that no new bugs are introduced in bugfix releases (caused by  
new features added meanwhile))

- maintain release branches for a while, at least until the next stable  
release (and port bugfixes back to mainline of course)

- create a series of release candidates before actually doing the  
release


some of those are very complex (peer review comes to my mind here)  
others not. I think we should at least require some of them


What do you think?



greets, Lars