[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [RFC] Automated QA build system

From: Richard Frith-Macdonald
Subject: Re: [RFC] Automated QA build system
Date: Thu, 24 Feb 2005 09:49:50 +0000

On 24 Feb 2005, at 05:39, Alex Perez wrote:

Fred Kiefer wrote:
I want to second Adam on this, especially as it was me who noticed the missing files in the first place. CVS is not the greatest of all version management tools, it is very easy to miss out on submitting single files. (You would know, if you were coding)

Is this supposed to be an insult? If so, it fell flat on its face. I've used CVS for years. Also, the GNUstep website itself is in CVS, which I update periodically on a fairly regular basis. Surely Richard has far more experience with CVS than I do.

I think he was just suggesting adopting a reasonable/patient attitude to CVS ... and that perhaps doing lots of cvs commits would help (of course, if yours never, ever go wrong, it's unlikely to promote tolerance ... but clearly he assumes yours would occasionally go wrong).

All of us really should try hard to avoid this from happening, but when it happens its fine to just correct it as soon as possible.

If you tested the actual code which gets committed (which IMHO is really a basline requirement) then this would not happen.

That's logical nonsense ... you can't test the code which gets committed until after it has been committed ... at which point the commit has already happened.

In this case the code was tested on three operating systems before the commit, (several cycles of test/fix before testing was ok) and the results of the commit itsself were tested (and fixed) afterwards (with a gap of a few hours as I had other things to do, but never the less, it was tested).

Things which break the entire build cycle should be taken more seriously.

Anyways, my comments were merely meant as constructive criticism. It's frustrating to check out CVS two times in a week and have it fail to build for such a trivial reason.

Yes ... but in large part the point of CVS is so you can get out different versions in the event of a problem at point in time. While I like to keep CVS as stable as reasonably possible, I do not believe that developers should waste a lot of time/energy on the issue ... I seem to recall that someone was doing regular snapshots of stable systems for people who didn't want to spend the few extra minutes needed to revert their checked-out state by a few hours when they happen to check out a broken version (which happens occasionally even when no mistakes have been make committing).

After a discussion on #GNUstep with Stefan Urbanek (stiivi), I will pass along his idea/suggestion. Some sort of automated QA mechanism would be great, where, upon a new CVS check-in to -base, for example, it would be checked out by an automated QA system and compiled on Linux/x86 as a baseline test. The results of the compilation would be placed on a webpage. If compilation breakage to any part of -core happened, a message would be sent to -dev notifying the core team that this had happpened, the offending file, and the version of the file in CVS at which the breakage occurred.

Does anyone think this would be valuable? It could be expanded upon/combined to do unit testing as well as pretty much any other Quality Assurance task you could dream up.

I doubt that there is a lot of point doing it for gnu/linux/x86 ... as that's what most (all?) the people doing work on the system will be testing on anyway ... it might notify us of a few problems quicker though ... which is a good thing as long as we don't rely on it as a substitute for the testing we are doing now (which would certainly be a temptation).

I'd appreciate hearing everyone's thoughts on this (that includes all of the core devs).

I'd also like to set this up for win32, since that seems to be the thing that breaks fairly often as other patches go into -core.

Yes ... for win32 and bsd (and perhaps solaris) this would be really useful, as these systems are infrequently used and a problem there can go unnoticed in cvs for a long time as most developers do not have the time (or the machines available) to test on these systems.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]