Re: [fluid-dev] Proposal: FluidSynth tester program

[David Henningsson]

On 07/12/2012 07:43 PM, jimmy wrote:
> On Wed, 11 Jul 2012 20:57, David Henningsson <address@hidden> wrote:
>
> > Something I've been thinking of for a while, and the recent
> > thread
> > reminded me of that thought...
> >
> > FluidSynth is quite a versatile program/library, and we all
> > want
> > different things out of it. No one of us has the full
> > picture, or uses
> > FluidSynth to all the different things it can be used for.
> > Making sure that none of all these use cases break, is one
> > of
> > FluidSynth's biggest challenges, and maybe sometimes it can
> > cause us to
> > be overly cautious.
>
> Same thing can be said for most softwares out there.  I agree about being 
> cautious, too.  I definitely don't want malware backdoors in any softwares, or 
> buggy softwares either.
>
> Although, being an open source community, it's the code contribution that make 
> the software grow.

Agreed.

> Imagine Linus writing all the changes alone, by himself, for the Linux kernel in it's 
> current form?  That would be insane.  I doubt that Linus even fully understand much of 
> what's in the hundreds of device drivers in the kernel, nor the hardware to test those 
> himself.  Can't even test all those changes even if he has all the hardwares working in 
> his lab(s), let alone having an "RC" (release candidate) available every 
> weekend for people to test.

I guess the difference here is dedicated time for development; there are 
hundreds or even thousands of people getting paid for working on the 
Linux every day, here we are only volunteers.

Linus has its subsystem maintainers which he trusts, but getting that 
trust would take years of good patch writing (I think).

> > Here's a proposal that might help us with that challenge.
> >
> >   . . .
> >
> > What do you think? It obviously won't be much of a tester
> > program
> > without a bunch of testers, so is this anything you think
> > would make so
> > much sense, that you would be willing to run a test or two
> > yourself?
>
> A 2-3 week testing period of some "release candidate" would help before a new 
> release anyway.
>
> People's hardware and software configurations can and will change as machine 
> break down, and replaced, as well as new releases of libraries, compilers...  
> It would be good to have a skeletal test-report to fill out, and posted, so 
> folks would have an idea of hardware/software, which distribution, 32/64-bit 
> OS...  Not sure if we need the library and compiler version number, but it 
> shouldn't hurt if those are reported back.

I guess this should be a bit adaptable depending on the test - for 
testing that it builds under a certain OS, compiler version might be 
more important than, say, testing envelope rendering or something like that.

> Some people might be on vacation, busy, out-sick...  But as folks in the FS-dev mailing 
> list subscribers, let's way every "release candidate" announcement should 
> encourage everyone here to do a test and report back.  Can't force anyone, but just say 
> we hope folks here will help track down any potential bugs anyway.
>
> Of course, any bug(s) found, confirmed, and fixed should result in a new "release 
> candidate" to be tested again, reset the testing time period.
>
> After a few rounds, we would have an idea of how many test-reports and what environments 
> each "release candidate" got tested, along with how many problems found...
>
> Not quite as formal as some automated test-suite, but would help along the way.

Yeah, I'm thinking somewhat along the same lines. Hopefully we won't 
have to make too many release candidates...

// David