discuss-gnustep
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Suggestion] GNUstep-test for quality control


From: Dennis Leeuw
Subject: Re: [Suggestion] GNUstep-test for quality control
Date: Thu, 16 Oct 2003 10:26:02 +0200
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020623 Debian/1.0.0-0.woody.1

Hi Stefan,

An important note: the Users of GNUstep are developers. GNUstep is a development environment. That's the problem.

So to get more Developers to develop programs with GNUstep you have to attracked them. One way to attracked new developers might be to provide next to a great API a great testing environment.

This way you can tell a developer when he runs into problems if it is his mistake (his app) or the GNUstep API. That might prevent them from running away from GNUstep and just file a bug report instead.

And automated mail-bug-script would be ideal.

Hope this helps,

Dennis

Stefan Urbanek wrote:
On 2003-10-15 22:58:38 +0200 Lars Sonchocky-Helldorf <address@hidden> wrote:

On 15.10.2003 20:58:37 Stefan Urbanek wrote:


<snip>


Simplier solution: What about lots of users as a 'testing framework and

quality

controll'? Advantage of 'user based testing framework' is that it tests continualy. That 'framework' should also contain users, that were marked

as

'kind of users that gnustep does not need' in a gworskpace thread few

days ago

on this list. More users means more testers and it does not matter what

kind of

users. At this time and state,  gnustep needs any kind of users. More

eyes see

more.

I think, that if at least half of the effort devoted to writing test

suites (i

am not against them) is also devoted to make gnustep more attractive, it

can

also help to remove bugs and have a quality controll.


That's a circular dependency: less bugs mean more users, more users mean less bugs. But the flipside is also true: lots of bugs mean less users, less users mean lots of bugs :-(. Currently the situation tends more to the latter...


Yes, and we need to get out of it.

The problem with a "unorganised user based testing" approach is that it needs committed, disciplined users to work. That's something you can't take for granted. My experience tells me instead that far most users simply turn away if things don't work out of the box: You even can be thankful if about one percent fire their mail client up and send you just complaints, the users that are able to give you a qualified bug report you can count with the fingers of both hands. Another result introduced by the nature of your approach is that you'll get many reports for easy bugs you already know, but virtually nobody is willing to invest a lot of time to track down a bug which a.) occurs irregulary or b.) involves a lot of work. In the end it is like in any chaotic multi agent system: the path of least resistance is chosen. The already long history of GNUstep is another proof for this.


True, users go away if it does not work. So what? We should at least have the first-user-experience parts bug-free, to let them go in and start play with the other stuff. Like a trap :-) You need a trap, hook or attraction to get users. Is there any in GNUstep? "The great framework desing with no alternative? Yes, but ... it does not work (as expected), so what?" You get the point.

About the discipline and organisation: you need that and you do not. Point is, that currently there is no need to fix bugs. Why? Because noone wants them to be fixed, or there are just a few people that want them to be fixed. So why waste an energy? More users means more pressure, and perhaps a chance that there is someone who will likely want to fix it. This is something about that famous "it works on my machine, so why bother?". True, why to bother, if there are just one or two users with this behaviour and noone else complains?

You make a mistake when you think that the "effort devoted to writing test suites" is basically lost time and done just for respectability. Let me tell you: it is different. Any amount of time invested in a proper test pays off. For a long time I also was a opponent of unit testing, I just saw no benefit from it, I resisted doing it and left it to the other guys here at work. But one day I *had* to do it and it came out to be much less complicated then I had feared. In fact I realized that I was so reluctant towards it because it was unknown to me. Today I have the habbit to test early and often. Unit-testing just helps you to make rapid progress while coding. I can tell you that I discovered lots of bugs, even 'thinkos' in my code doing testing (often borderline cases where some special treatment for certain values was needed) that would have gone undiscovered and would have caused bugs hard to track down later on (if you build on a faulty foundation you building will most likely be faulty to). I wouldn't start a project (even a private) without unit testing today.


No, I haven't said that the effort devoted to writing test is lost time, just that at least half of the time should be devoted to gaining more users. This is not a commercial software for particular requestor where you already have a customer, this is free software where "customers" are not certain. And if you write a SW that is really perfect (according to tests), why is it good for if it is for noone just those who developed it? Then the SW will die. I'm saying that testing by users can be complimentary to testing with a testing framework. GNUstep needs both. Now a question is, what is easier to get at this time?

Btw., to paraphrase your last question ... Would you start a project without users (even potential) today? Are there going to be any future users with this state and attitude of GNUstep?

Moreover, there are functionalities that can be tested only by users, so saying that users are not good for testing is not a way to go.

To sum it up:
- user testing is complimentary to unit testing
- more users will result in greather pressure/need of bug fixing
- to get more users, there should be a trap/hook/attraction - is there any?
- without users, project dies with its developers

Have a nice day,

Stefan Urbanek







reply via email to

[Prev in Thread] Current Thread [Next in Thread]