Re: splinefit test failures

[Daniel J Sebald]

On 08/02/2012 01:24 PM, Rik wrote:
> On 08/02/2012 10:45 AM, Daniel J Sebald wrote:
> > > The benchmark sum shows 898 so approximately 10% of the time the tests
> > > fail.  On looking through the results in the tst_spline.err log I see that
> > > it is only when randn has returned a value exceptionally far from the
> > > expected mean of 0 do we get a test failure.  Given that randn can return
> > > any real number between -Inf, +Inf we might be better testing the function
> > > with a narrower input.
> > >
> > > Replacing
> > > %! yb = randn (size (xb));  range is [-Inf, Inf]
> > > with
> > > %! yb = 2*rand (size (xb)) - 1; range is [-1,1]
> > >
> > > changes the success rate to 999/1000.
> >
> >
> > Seems a worthwhile test to me.  Again, I'd loosen the tolerance a bit to
> > the point of making random deviations outside of tolerance small.  And if
> > that doesn't happen, then it needs to be fixed.
> >
> > Dan
> E-mail is not a very clear communication medium.  Is your vote to keep
> randn, drop the initialization to a specific seed, and loosen the
> tolerance?  Or is it to switch to rand()?

Drop the specific seed, use random.  Try several (hundred/thousand) 
repeated tests to explore the space.

Loosen tolerance on the order of 100*eps.  Maybe that is too big, but I 
think a good point is somewhere between 10*eps and 100*eps.  Seeing the 
results I have, 10*eps feels like it is within random deviation.

randn/rand, not sure.  rand() will certainly get rid of large deviations 
as an input.  But maybe the issue is the tolerance too small.  I'd start 
with trying a tolerance of 30*eps.  Maybe that will obviate the 
randn/rand problem.

Dan