bug-gnubg
[Top][All Lists]

Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13

 From: Rod Roark Subject: Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13 Date: Mon, 9 Jun 2003 07:14:08 -0700 User-agent: KMail/1.4.3

```As others mentioned, there's a square root involved in the
calculation of standard deviation.  I believe there are a
couple of different ideas in use as to the exact formula.

However I don't think you want this calculation anyway.  It
is an attempt to answer the question "how much do the
results differ from the average?".  Well if each result
must be either 0 or 1 and the average is somewhere around
0.5, then they will all differ by 0.5.  This is not an
interesting number.

The question to answer is, "how significant is the result
from 100 matches?".  This depends in part on "how much does
a match depend on skill, and how much on luck?".  The answer
to that last question is elusive.

I think what might be necessary is to run many batches of
100 matches each, and then compute the average and standard
deviation of the results of those.

However it's been many years since I studied probability
theory, and others may have more insight than I.

-- Rod

On Monday 09 June 2003 03:14 am, Joern Thyssen wrote:
> On Mon, Jun 09, 2003 at 12:12:23AM -0300, Albert Silver wrote
>
> > Hi all,
> >
> >     This is just to report that the series of 100 7-point matches
> > has finished and the final score was 56-44 in GNU's favor. Attached are
> > the matches so that any may see them. Many thanks to Tony Lezard for his
> > program that made the match-up possible, and of course a huge
> > congratulations to the GNU team.
>
> Can someone help me to calculate the std. deviation, please!
>
> My idea was to calculate gnubg's average win, e.g.,
>
>     sum(i=1,100) result_i
> p = ---------------------
>          100
>
> where result_i=1 if gnubg won and result_i=0 if Snowie won.
>
> The variance can be written as
>
> variance=average(result^2) - ( average(result) ) ^2
>
> However, as average(result^2) = average(result)=p then the variance can b
> reduced to
>
> variance=p-p^2
>
> That, is /independent/ of the number of samples!
>
> In this example I get
>
> mean=0.56 and std. deviation=0.50, thus
>
> gnubg wins 0.56 +/- 0.98
>
> What am I doing wrong?
>
> Jørn

```