Re: [Bug-gnubg] An evalutaion of the pruning nets

[Jim Segrave]

On Mon 25 Oct 2004 (08:13 +1300), Joseph Heled wrote:
> 
> Jim,
> Thanks for the detailed analysis. I did some tests when developing the 
> pruning, but not in such scope.
> But I wonder, since 2,375 is not a big number, can you publish it in some 
> simple text format (Say "positionID matchID dice" per line), bzip it and 
> send it to me?
> (Same for differing cube decisions, if any)


What's easy to do is get the individual lines from the .sgf file (or
even simply identify the .sgf file, game number and move number where
the differences occur.

What I could do is let you pick up the actual .sgf files and the file
listing the moves and the errors seen from my machine - you still have
an account, so I could prepare a .tar.gz or a zip file of all the
data for you to scp off. The error file from my script identifies the
file, game no within the match and move number within the game (plus
or minus one or so). Since it also includes the actual lines from the
SGF file, it's possible to be sure you are looking at the right one -
they begin with ;B[31abcd] for example if B (player 1) is moving and
the dice are 31.

To get position Ids and match Id's I'd either have to manually load up
the match and step to the right game and move or add a lot more
parsing code. 

I don't have any doubling information, as I didn't feel like writing
the extra perl to handle that, but since doubling decisions are based
on the evaluator and the MET, if we think the evaluator hasn't changed
much, them the cube will be the same.

When I just fixed my script to give game numbers within the match and
moves within the game, I discovered that my reported results were
wrong - there are 297 matches, but only 79466 moves ( the old script
had a line which should have been deleted which added an extra count
for each move as it was read from each file, so I ended up with a
count 3 times the real value. This makes the standard errors a bit
larger

297 matches, 2509 games, 79466 moves
3776 forced moves, 569592 positions
The best move was different in 2375 cases
Average Diff/Std Err
0.00123 0.00108 0.00009 0.00157 0.00020 0.00343 
0.00619 0.00390 0.00051 0.00642 0.00403 0.02100 

And means the percentage of re-ordered moves is about 2.98% of the
cases, not 0.94.


-- 
Jim Segrave           address@hidden