Re: [Bug-gnubg] Re: New Contact Net Error Rates

[Jim Segrave]

On Thu 20 Feb 2003 (12:02 -0500), Moore, Dave wrote:
> Jim Segrave wrote:
> 
> >This leads me to a point I've been wondering about. Everyone discusses
> >gnubg having errors on odd ply evaluations. Can someone explain why
> >this would be? I have always taken it on faith that this is true, but
> >I'd like to understand the mechanism.
> 
> 
> I have read on this list that it is because at odd-ply evaluations the NN is
> evaluated from the other player's point of view, and this gives different
> absolute equity results for some positions.
> 
> My naive interpretation:
> 
> For example, a position viewed from player 0's point of view may evaluate to
> +0.600 equity, but the same position evaluated from the other side will
> evaluate to -0.589 equity.
 
But it doesn't evaluate the same positions from the other side's point
of view, it evaluates what the other side would thing if a particular
position arises. I would (naively) expect this to be a problem only if
some of the neural net outputs are significantly less accurate than
others - for example if the NN is very good at estimating w/g/bg but
makes larger errors in the lose/lg/lbg portion, then the "other
player's point of view" would make sense to me.
 
> I also have two naive questions:
> 
> 1.  Wouldn't it be possible to run the odd-ply evaluations while always
> evaulating the board from player 0's point of view?  You would still go
> through the possible dice and moves for player 1, but the move would be
> selected by evaluating the resulting position from player 0's point of view,
> always.  This would eliminate the jumps in absolute equity numbers.

Unless I'm wrong, which is quite possible, that's what happens
now. You start with a 0 ply evaluation of a position and a dice
roll. For each possible move, you get a NN evaluation of the resulting
position. You select the most promising of these, and for each one,
you now look at the opponent's possibilities - every possible roll and
the resulting evaluation for the opponent. From this, you work out
what the equity is for the original player - eg. 

player 0 has a 6-1 opening. 
you evaluate, among others 13-7 8-7.

you now look at what player 1 would be doing given that opening
move. Let's say over all rolls, player 1 has

   win   gam  bkg  lose lose g lose bg
   .450 .185 .002  .550 .216    .004

Would this not imply that player 0 playing 13-7 8-7 has an expected
result (1 ply) of

   win   gam   bkg  lose lose g lose bg
  .550  .216  .004  .450 .185    .002

(in other words the one ply results with wins and lossees reversed)

The neural net evaluations aren't in and of themselves ply dependant,
they are static evaluations of a single position. 

-- 
Jim Segrave           address@hidden