Re: [Bug-gnubg] I'm a bit confused

[Joern Thyssen]

On Wed, Jul 30, 2003 at 12:07:13AM +0200, Jim Segrave wrote
> > 
> > You may see similar problems for evaluations as well, but as evaluations
> > are never truncated then the problem is usually not so large.
> 
> Now I'm confused - I would think that the wins/gammons/backgammons
> figures are accurate, but probably not complete enough. In the
> situation you describe where you'll be doubled out, your wins should
> be 0.0, lose gammon and lose backgammon will be either 0.0 or 1.0
> depending on where your pieces are.

NO! If the rollout is truncated at "double, pass" gnubg uses the
cubeless gwcs from the cube decision leading to "double, pass", It does
*NOT* use 1 0 0 0 0!!!  "1 0 0 0 0" is a *cubeful* gwc and we can't mix
that with the cubeless gwcs from the other trials.

My example was a bit wrong:

Worst case scenario: you're on roll in a position where you'll be
doubled out no matter what you roll. If you do a 0-ply cubeful rollout
then the cubeful equity is -1 and the cubeless gwc is identical to a
cubeless 0-ply evaluation even though you've selected 1,679,616 trials.
^^^^^^^^^^^^^^

 GNU Backgammon  Position ID: tm0DBwDgc/ABMA Match ID   : cIkIAAAAAAAA
 +13-14-15-16-17-18------19-20-21-22-23-24-+     O: gnubg
 | X              O |   | O  O  O  O  O  X |     0 points
 | X              O |   | O  O  O  O  O  X |     
 | X                |   |                  |     
 | X                |   |                  |     
 | X                |   |                  |    
v|                  |BAR|                  |     (Cube: 1)
 |                  |   | X                |    
 |                  |   | X                |     
 | O           X    |   | X                |     
 | O           X    |   | X                |     Rolled 12
 | O           X    |   | X                |     0 points
 +12-11-10--9--8--7-------6--5--4--3--2--1-+     X: jth

0-ply cubeful rollout:

    1. Rollout          6/5 6/4                      Eq.:  -1.000
       0.211 0.017 0.000 - 0.789 0.295 0.016 CL  -0.872 CF  -1.000
      [0.000 0.000 0.000 - 0.000 0.000 0.000 CL   0.000 CF   0.000]
        Truncated cubeful rollout (depth 11) with var.redn.
        1296 games, Mersenne Twister dice gen. with seed 1175268096 and
quasi-random dice
        Play: 0-ply cubeful [expert]
        Cube: 0-ply cubeful [expert]
    2. Rollout          8/7 6/4                      Eq.:  -1.000 (
+0.000)
       0.203 0.016 0.000 - 0.797 0.293 0.015 CL  -0.887 CF  -1.000
      [0.000 0.000 0.000 - 0.000 0.000 0.000 CL   0.000 CF   0.000]
        Truncated cubeful rollout (depth 11) with var.redn.
        1296 games, Mersenne Twister dice gen. with seed 1175268096 and
quasi-random dice
        Play: 0-ply cubeful [expert]
        Cube: 0-ply cubeful [expert]

0-ply cubeless:

    1. Cubeless 0-ply   6/5 6/4                      Eq.:  -0.872
       0.211 0.017 0.000 - 0.789 0.295 0.016
        0-ply cubeless
    2. Cubeless 0-ply   8/7 6/4                      Eq.:  -0.887 ( -0.015)
       0.203 0.016 0.000 - 0.797 0.293 0.015
        0-ply cubeless

Exactly the same gwcs as I predicted!

> I would find the cubeless rollout figures just as strange, since they
> would show results which would never happen when the cube is live -
> even in a situation where you're going to be doubled out, 

No, since gnubg would play to optimize the cubeless equity rather than
the cubeful one.

> If rollouts of moves kept statistics, then it would be useful to look
> and see where the wins and losses actually occur, since that's the
> only reason I can see for Albert's puzzling results. Certainly many
> users of gnubg, snowie, jellyfish et. al. routinely look at the
> win/gammon/backgammon figures when trying to understand why the bots
> prefer one move over another.

Yes, and that's precisely why I more than once have thought about
removing the cubeless gwcs from cubeful rollout and evaluations: they
are simply not reliable, so you can reach wrong conclusions -- and worst
of all: we'll see questions like Albert's again and again from confused
users!!!

Jørn