RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13

[Ian Shaw]

> -----Original Message-----
> From: address@hidden
> [mailto:address@hidden Behalf Of Joern
> Thyssen
> Sent: 09 June 2003 15:29
> To: Rod Roark
> Cc: GnuBG List
> Subject: Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13
> 
> 
> On Mon, Jun 09, 2003 at 07:14:08AM -0700, Rod Roark wrote
> > As others mentioned, there's a square root involved in the
> > calculation of standard deviation.  I believe there are a
> > couple of different ideas in use as to the exact formula.
> > 
> > However I don't think you want this calculation anyway.  It
> > is an attempt to answer the question "how much do the
> > results differ from the average?".  Well if each result
> > must be either 0 or 1 and the average is somewhere around
> > 0.5, then they will all differ by 0.5.  This is not an
> > interesting number.
> > 
> > The question to answer is, "how significant is the result
> > from 100 matches?".  This depends in part on "how much does
> > a match depend on skill, and how much on luck?".  The answer
> > to that last question is elusive.
> 
> I know how to calculate the luck by analysing the matches and 
> extracting
> the total luck. 
> 
> The 95% confidence interval is calculated as 1.96 * std.dev. 
> So with 100
> matches and a final result of 56 wins to gnubg and std.dev of 0.5, I
> get:
> 
> 56% +/- 98%
> 
> What I don't understand is that if I had played 100000000000 
> matches and
> gnubg won 54000000000 of them, I would still have a std.dev 
> of 0.5, thus
> I still get:
> 
> 56% +/- 98%
> 
> So what did I get wrong?
> 
> Jørn
> 
Perhaps the numbers are right and it's out interprettion that's wrong.

The numbers say that the average reault is 0.56 in favour of GnuBg, +-0.98 
95%CI, assuming a normal distribution. It's certainly true that the results 
fitted in the range 1.54 to -0.42, but the distribution wasn't a normal one.

I think Rod is perhaps right, and we need stats from several series of matches.

OTOH, maybe not. Check out these threads from rec.games.backgammon:

http://www.bkgm.com/rgb/rgb.cgi?view+709 (Some of the names may be familiar ;)
http://www.bkgm.com/rgb/rgb.cgi?view+248

Maybe the question can be posed as: If GnuBg and Snowie 4 are equal players, 
what is the likelihood of 
GnuBg winning 56 out of 100?

--Ian