Re: [Bug-gnubg] Verification of hypergammon database

[Christopher Yep]

Hi,

I recently started playing around with hypergammon.  After getting some 
strange results, I found this e-mail sent to the mailing list on 1/18/2007 
(below).

I was able to exactly duplicate Nikolay's results for recent versions of 
gnubg (from 2006 and 2008), but I got correct results for a version of 
gnubg from 2004.  This suggests that something may have changed with the 
hypergammon-3 evaluator (or at least how gnubg interprets the results of 
the hypergammon-3 evaluator) between 2004 and 2006.

Here is the position referenced by Nikolay (Jacoby rule active):

    GNU Backgammon  Position ID: ADAAAQAkIAAAAA
                    Match ID   : cAkAAAAAAAAA
    +24-23-22-21-20-19------18-17-16-15-14-13-+  O: gnubg
    | X                |   |    X     X       |  0 points
    |                  |   |                  |
    |                  |   |                  |
    |                  |   |                  |
    |                  |   |                  |
    |                  |BAR|                  |v (Cube: 1)
    |                  |   |                  |
    |                  |   |                  |
    |                  |   |                  |
    |                  |   |                O |  On roll
    |    O             |   |                O |  0 points
    +-1--2--3--4--5--6-------7--8--9-10-11-12-+  X: user

I also did a quick test of a hypergammon-2 position (using a 2008 version 
of gnubg), which also gave incorrect results.  (I didn't check any 
hypergammon-2 positions with my version of gnubg from 2004).  So, the 
problem probably also extends to hypergammon-2 and hypergammon-1.

I was able to duplicate Nikolay's result for two recent versions of gnubg:

Windows XP, GUI multiple-thread version, version 0.90-mingw (build Apr 25 2008)
Windows XP, GUI single-thread version, version 0.16-mingw (build Nov 20 2006)

However, I do get correct results for a version of gnubg from 2004:

Windows XP, GUI (single-thread) version, version 0.14-devel (build Apr 20 2004)

The correct win/gammon/bg numbers are:

45.6%, 24.4%, 1.4%; 54.4%, 31.8%, 1.9% instead of 81.8%, 49.4%, 8.7%; 
18.2%, 0.0%, 0.0%.  Obviously the second set of numbers is unreasonable 
(with O supposedly winning only 18.2% of the time with zero gammons).

The correct cubeful equities are ND = -.219, D/T = -.710 (i.e. 2 * 
-0.3548), so the correct cube action is no double, beaver.  My 2004 version 
of gnubg gets everything correct.  My 2006 and 2008 versions of gnubg have 
problems.  If one tries to play the above position with tutor mode on (with 
my 2006 or 2008 versions), gnubg will complain if X fails to double (since 
gnubg claims that ND = .798 and D/T = 2.381, i.e. gnubg claims that it's a 
blunder not to double and that it's a huge pass).

Chris

At 05:02 AM 1/18/2007, Nikolay Marinov wrote:
> Hi All,
>
>    I have problem with hyper gammon database for 3 chequer backgammon. I 
> generate it myself and also download a generated copy from the internet, 
> but when I try to verify this databases with position
>
>         GNU Backgammon  Position ID: ADAAAQAkIAAAAA
>                         Match ID   : cAkAAAAAAAAA
> given in gnubg site I've got the following result:
>
> Position ID:    ADAAAQAkIAAAAA
> Match ID:       cAkAAAAAAAAA
>
> Evaluator:      HYPERGAMMON-3
>
>             Player       Opponent
> Position         3018          2831
>
> Owned cube                    : -0,0230
> Centered cube                 : -0,2310
> Centered cube (Jacoby rule)   : -0,2186
> Opponent owns cube            : -0,3548
>
>        Win     W(g)    W(bg)   L(g)    L(bg)   Equity    Cubeful
> static: 0,818   0,494   0,087   0,000   0,000    +1,218    +0,798
>
>
>
> Cube analysis
> 0-ply cubeless equity  +1,218
>  0,818 0,494 0,087 - 0,182 0,000 0,000
> Cubeful equities:
> 1. Double, pass         +1,000
> 2. Double, take         +2,381  ( +1,381)
> 3. No double            +0,798  ( -0,202)
> Proper cube action: Double, pass
>
> which is slightly different from evaluation of the same position given in 
> the site. I try this in 0.14 and 0.15 version of gnubg on Windows XP SP2, 
> but both returns the same result.
>
> Do You have any idea from where this problem may come? Thaks in advance.
>
> Nikolay.