Re: [Bug-gnubg] Re: Naw..panic or teach me what I am doing wrong !!

[Jim Segrave]

On Wed 08 Jan 2003 (21:45 +0100), Jim Segrave wrote:
> On Wed 08 Jan 2003 (18:32 +1300), Joseph Heled wrote:
> > 
> > There seems to be a discrepancy. My code gives me consistently
> > 
> > 0.54213905334472656, 0.34824475646018982, 0.08864891529083252, 0.0, 0.0
> > 
> > When I use gnubg, I get Neil numbers or worse.
> > 
> > Have no idea yet what is wrong.
> > 
> > Can gnubg log the rollouts? I would like to analyze them with fibs2html 
> > to see where the differences are?
> 
> I was thinking the same thing. It's easy to syslog the dice rolls, the
> game number, player on move, cube action (although it's not important
> here). The hard part is that all rollouts seem to use
> BasicCubefulRollout() and that doesn't actually provide the chosen
> move. 
> 
> I can change the call to FindBestMove() in this routine to return the
> best move for non 0ply rollouts. Fixing it for 0 ply is a bit messy.
> 
> These patches may be a bit ugly and will assume the presence of
> syslog() and openlog(), so it may not be usable under Windows.

OK - I've attached a context diff which (under Unix) will log to
loglocal4.notice. I won't check this in now, as I think it's only a
debugging tool and not production code. It won't work with 0 ply
rollouts and I haven't tested it's cube handling. Truncated rollouts
will require a fair bit of work to evaluate results. In fact, even
with untrucated results, it's left as an exercise to the reader to
figure out who won.

To use it, you will need to add something like this line to
/etc/syslog.conf:

local4.*                                        /var/log/bg.log

(use tabs, not spaces), then kill -HUP the syslog PID. For rollouts,
you will get something like this in /var/log/bg.log or wherever you
tell syslog to put it:

Jan  9 00:40:40 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 0: set dice 55
Jan  9 00:40:43 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 0: 5/0 5/0 
5/0 5/0 
Jan  9 00:40:43 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 1: set dice 42
Jan  9 00:40:43 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 1: 22/18 
18/16 
Jan  9 00:40:43 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 2: set dice 32
Jan  9 00:40:46 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 2: 6/3 6/4 
Jan  9 00:40:46 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 3: set dice 65
Jan  9 00:40:46 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 3: 25/20 
16/10 
Jan  9 00:40:46 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 4: set dice 55
Jan  9 00:40:48 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 4: 4/0 4/0 
4/0 4/0 
Jan  9 00:40:48 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 5: set dice 54
Jan  9 00:40:48 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 5: 24/19 
18/14 
Jan  9 00:40:48 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 6: set dice 63
Jan  9 00:40:51 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 6: 4/0 3/0 
Jan  9 00:40:51 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 7: set dice 52
Jan  9 00:40:51 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 7: 19/14 
20/18 
Jan  9 00:40:51 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 8: set dice 22
Jan  9 00:40:52 allium-ursunum gnubg rollout[55571]: Game: 0 Turn 8: 2/0 2/0 
2/0 

You can reduce this to valid commands by running the log file through
grep and sed:

grep 'Game: 0 ' bg.log | sed -e 's/.*Turn [0-9]*: //' | tee /tmp/n

will extract only the game 0 commands and strips all the syslog cruft
away:

set dice 55
5/0 5/0 5/0 5/0 
set dice 42
22/18 18/16 
set dice 32
6/3 6/4 
set dice 65
25/20 16/10 
set dice 55
4/0 4/0 4/0 4/0 
set dice 54
24/19 18/14 
set dice 63
4/0 3/0 
set dice 52
19/14 20/18 
set dice 22
2/0 2/0 2/0 

I will leave it running out 99 games tonight.
The above game already is interesting. It was done with the rollout at
2 ply with filters:

      Move filter for 2 ply:
        keep the first 2 0-ply moves and up to 3 more moves within
             equity 0.1
        keep the first 0 1-ply moves

When analysed 2 play with 
      Move filter for 2 ply:
        keep the best 0-ply move and up to 10 more moves within equity 0.2
        keep the best 1-ply move and up to 5 more moves within equity 0.2

player 0 (the backgame player) gets an interesting score sheet (I
didn't have cubeful analysis turned off):

                                      gnubg2pX             gnu2pO
        
Checkerplay statistics:
        
Total moves                           4                    5
Unforced moves                        4                    2
Moves marked very good                0                    0
Moves marked good                     0                    0
Moves marked interesting              0                    0
Moves unmarked                        3                    5
Moves marked doubtful                 1                    0
Moves marked bad                      0                    0
Moves marked very bad                 0                    0

Error rate (total)                    -0.055 (-0.055)      -0.000 (-0.000)     
Error rate (pr. move)                 -0.014 (-0.014)      -0.000 (-0.000)     
Checker play rating                   Expert               Supernatural        

Rolls marked very lucky               0                    1
Rolls marked lucky                    0                    0
Rolls unmarked                        2                    2
Rolls marked unlucky                  0                    2
Rolls marked very unlucky             2                    0
        
Luck rate (total)                     -1.924 (-1.924)      +0.540 (+0.540)     
Luck rate (pr. move)                  -0.481 (-0.481)      +0.108 (+0.108)     
Luck rating                           Haaa-haaa            Go to Las Vegas imme

Cube decision statistics:
        
Total cube decisions                  4                    4
Actual or close cube decisions        0                    4
Doubles                               0                    0
Takes                                 0                    0
Pass                                  0                    0

Missed doubles around DP              0 (-0.000, -0.000)   1 (-0.194, -0.194)  
Missed doubles around TG              0 (-0.000, -0.000)   0 (-0.000, -0.000)  
Wrong doubles around DP               0 (-0.000, -0.000)   0 (-0.000, -0.000)  
Wrong doubles around TG               0 (-0.000, -0.000)   0 (-0.000, -0.000)  
Wrong takes                           0 (-0.000, -0.000)   0 (-0.000, -0.000)  
Wrong passes                          0 (-0.000, -0.000)   0 (-0.000, -0.000)  
Error rate (total)                    -0.000 (-0.000)      -0.194 (-0.194)     
Error rate (per cube decision)        -0.000 (-0.000)      -0.049 (-0.049)     
Cube decision rating                  n/a                  Beginner
        
Overall
        
Overall error rate (total)            -0.055 (-0.055)      -0.194 (-0.194)     
Overall error rate (per decision)     -0.014 (-0.014)      -0.032 (-0.032)     
Overall rating                        Expert               Beginner
       
I reran the moves and did an analysis without chequer play by score
and cubeful analysis off and still player 0 gets an overall error of
0.62 average 0.015. The rollout settings were the same for user 0 and
user 1.

This is a game that the backgame player loses badly (to a
backgammon). I'll leave it to dump 100 rollouts tonight.


-- 
Jim Segrave           address@hidden

roll.diff
Description: Text document