Re: [gnugo-devel] SlugGo v.s. Many Faces

[David G Doshay]

On Aug 24, 2004, at 10:22 AM, Douglas Ridgway wrote:

> On Mon, 23 Aug 2004, David G Doshay wrote:
>
> > [game results...]
> > So, SlugGo wins about 20 out of 25 from Many Faces. We will continue
> > this contest against Many Faces until we have 100 games.
>
> Very interesting. I took the liberty to run some numbers on this data 
> (*).
> ... It seems clear that SlugGo is more than 1/2 stone better
> than ManyFaces (p = 0.0053 to get this by flipping coins).
>
> Given that, I don't understand the plan to play more games at a 
> handicap
> of 0.5.

Some of it was an attempt to be sure that our numbers are significant, 
but
now that you have done the statistics for us, we can be confident of 
that.
Thanks. Some of it is just the fun of winning; simple positive feedback 
after
a year of work. ;^)

>  If we take the average margin of 31 as indicative of the handicap
> level, and use 2*(komi 6.5) as the value of each stone, we estimate 
> SlugGo
> as 2.38+0.5 = 2.88 stones better than Many Faces. If this is right, you
> ought to be able to show a statistically significant margin at a 1.5 
> stone
> handicap (2 stones, no komi) within a hundred games, probably less. 3
> stones is probably the fair handicap, but it would likely require 
> hundreds
> of games to determine who was better at three stones.
>
> So that'd be my vote. Give two stones to Many Faces, or take on another
> top program, or play against people.

We can try 2 stones against MFG and see what happens. We would like to
try playing a series of games against Go++, but that will be very hard 
to
automate. We had to do a bit of work just to automate a series of games
against MFG.

We do plan to start a series of games against people, but they will 
have to
be patient people. At the strongest levels we have found so far, it 
runs about
1/18th the speed of GNU Go.

But before we do either there are some changes I would like to make in 
the
code. I am not happy with the first 5 or so moves and need to implement 
a
fix.

> (*) I only see 24 games here, and I threw out the game with the unclear
> result, so I worked with 23 games.

I was copying results by hand, so I must have slipped. Sorry. The missed
entry is:
        White wins by 25
from the first set of 15, for which a link to the sgf files is given.

Cheers,
David