Re: [fluid-dev] Proposal: FluidSynth tester program

[jimmy]

On Wed, 11 Jul 2012 20:57, David Henningsson <address@hidden> wrote:

> Something I've been thinking of for a while, and the recent
> thread 
> reminded me of that thought...
> 
> FluidSynth is quite a versatile program/library, and we all
> want 
> different things out of it. No one of us has the full
> picture, or uses 
> FluidSynth to all the different things it can be used for.
> Making sure that none of all these use cases break, is one
> of 
> FluidSynth's biggest challenges, and maybe sometimes it can
> cause us to 
> be overly cautious.

Same thing can be said for most softwares out there.  I agree about being 
cautious, too.  I definitely don't want malware backdoors in any softwares, or 
buggy softwares either.

Although, being an open source community, it's the code contribution that make 
the software grow.

Imagine Linus writing all the changes alone, by himself, for the Linux kernel 
in it's current form?  That would be insane.  I doubt that Linus even fully 
understand much of what's in the hundreds of device drivers in the kernel, nor 
the hardware to test those himself.  Can't even test all those changes even if 
he has all the hardwares working in his lab(s), let alone having an "RC" 
(release candidate) available every weekend for people to test.


> Here's a proposal that might help us with that challenge.
> 
>  . . .
> 
> What do you think? It obviously won't be much of a tester
> program 
> without a bunch of testers, so is this anything you think
> would make so 
> much sense, that you would be willing to run a test or two
> yourself?

A 2-3 week testing period of some "release candidate" would help before a new 
release anyway.

People's hardware and software configurations can and will change as machine 
break down, and replaced, as well as new releases of libraries, compilers...  
It would be good to have a skeletal test-report to fill out, and posted, so 
folks would have an idea of hardware/software, which distribution, 32/64-bit 
OS...  Not sure if we need the library and compiler version number, but it 
shouldn't hurt if those are reported back.

Some people might be on vacation, busy, out-sick...  But as folks in the FS-dev 
mailing list subscribers, let's way every "release candidate" announcement 
should encourage everyone here to do a test and report back.  Can't force 
anyone, but just say we hope folks here will help track down any potential bugs 
anyway.

Of course, any bug(s) found, confirmed, and fixed should result in a new 
"release candidate" to be tested again, reset the testing time period.

After a few rounds, we would have an idea of how many test-reports and what 
environments each "release candidate" got tested, along with how many problems 
found...

Not quite as formal as some automated test-suite, but would help along the way.

Jimmy