Re: [Bug-gnubg] Bot comparison 2-ply v 2-ply pruned.

[Jim Segrave]

On Tue 27 Apr 2010 (06:44 +0000), Michael Depreli wrote:
> 
> Hi Jim
> What I need is the following:
> I have 50 money game sessions (48*10 games and 1*20 games).
> I need the script to compare the sessions and output the following where 
> there are differences:
> 
> Filename: 
> Game#: 
> Move# :
> Player: 
> Cube And/Or Checker Play:  (It's important that the script doesn't overlook 
> when a cube error and checker error occur on the same move)
> 
> I don't need to know the equity differences etc because I will be using my 
> own rollout data but it may obviously be useful for others.
> 
> Also I see from the mailing list that there are moves afoot to strengthen the 
> net so if we wanted to compare the old net vs the new one on my benchmark, 
> this tool would be very useful.
> 
> I'm really not familiar with Perl etc, so if you could write me a script and 
> run the sessions through for me it would be a great help

Put the .sgf files into two zipfiles one where the analysis was done
one way, one where it was done the other way. You can mail teh zip
files to me (there's essentially no limit on the size of mail I can
receive at address@hidden).

I'll run them trhough my script and report back by game number and
move number where the nalyses differ both in selecting what they think
is the best move and where the analysis of any of the possible moves
differed.

I don't currently hva cube handling in there - this was written to
compare the pruning net when it first came out with the non-pruning
version to see if pruning was affeting the evaluations. I'll have to
look to see what it would take to compare cube decisions. However, cub
decisions are made based on the neural net evaluations and some
external functions not related to the neural net, so differences in
evaluaing moves will be very indicative of differences in cube play

I'll be on holiday until mid-May, so you won't get a real quick answer,


-- 
Jim Segrave           address@hidden