Re: [Bug-gnubg] pubeval benchmark

[Mark Higgins]

Ah ha! I think I know what's wrong with your implementation, Joseph.

If I flip the perspective of the board before calculating the pubeval values 
for a given move, I get exactly what you have below.

And the performance of pubeval using the "flipped board" calculation is much 
worse than with the regular calculation, so I'm pretty sure it's an error.



On Jan 19, 2012, at 9:57 AM, Joseph Heled wrote:

> Here are all the possibilities for a 3,1 from the initial position. I
> am worried that 8-5 6-5 sneaks at number 6
> 
>   score               gnubg id                       position
> 8.00384996831 4HPwATDgc/ABIQ (0, 1, 0, 0, 0, 1, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0)
> 7.83777993917 4HPwATDEc/ABKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 3, 0, 4, 0, 0, 1, 0, -2, 0)
> 7.81063996255 4HPwATDgc/ABEg (0, 0, 1, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0)
> 7.75730997324 4HPwATDgc+IBKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
> -5, 0, 1, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0)
> 7.43968993425 4HPwATDQZ/ABKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 2, 0, 5, 1, 0, 0, 0, -2, 0)
> 6.84054994583 4HPwATCwZ/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 2, 0, 4, 2, 0, 0, 0, -2, 0)
> 6.81688992679 4HPwATDga/ABIg (0, 1, 0, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 2, 1, 5, 0, 0, 0, 0, -2, 0)
> 6.67967991531 4HPwATDQc/ABIg (0, 1, 0, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 3, 0, 4, 1, 0, 0, 0, -2, 0)
> 6.67962995172 4HPwATDCc/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 3, 0, 4, 0, 0, 0, 1, -2, 0)
> 6.24834996462 4HPwATDIZ/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 2, 0, 5, 0, 1, 0, 0, -2, 0)
> 6.10885995626 4HPwATDgc+EBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
> -5, 0, 0, 1, 3, 0, 5, 0, 0, 0, 0, -2, 0)
> 4.3719098568  4HPwATDEa/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 2, 1, 4, 0, 0, 1, 0, -2, 0)
> 4.29143989086 4HPwATDga+IBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
> -5, 0, 1, 0, 2, 1, 5, 0, 0, 0, 0, -2, 0)
> 4.23469984531 4HPwATCkc/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 3, 0, 3, 1, 0, 1, 0, -2, 0)
> 4.15422987938 4HPwATDQc+IBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
> -5, 0, 1, 0, 3, 0, 4, 1, 0, 0, 0, -2, 0)
> 3.22565984726 4HPwATDQV/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
> -5, 0, 0, 0, 1, 1, 5, 1, 0, 0, 0, -2, 0)
> 
> 
> On 20 January 2012 03:45, Mark Higgins <address@hidden> wrote:
>> :) I'll give it a try and check the couple games you sent.
>> 
>> 
>> On Jan 19, 2012, at 9:39 AM, Joseph Heled wrote:
>> 
>>> I was asking for help, not for more work.
>>> 
>>> For example, anyone who implemented play against pubeval can check a
>>> few of the moves I sent.
>>> 
>>> -Joseph
>>> 
>>> On 20 January 2012 03:35, Mark Higgins <address@hidden> wrote:
>>>> I used the following post to define pubeval; is this the proper definition 
>>>> still:
>>>> 
>>>> http://www.bkgm.com/rgb/rgb.cgi?view+610
>>>> 
>>>> ?
>>>> 
>>>> 
>>>> 
>>>> On Jan 19, 2012, at 8:31 AM, Joseph Heled wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I just run 10000 1 point match games between gnubg and pubeval. I
>>>>> think this is the most fair, since there are no gammons of cubes
>>>>> involved.
>>>>> 
>>>>> However, gnubg won 9840 of those, which makes me think something is wrong.
>>>>> 
>>>>> Can someone help me verify that pubeval makes the "right" moves?
>>>>> 
>>>>> I can send a few games in oldmoves format, or can check specific moves.
>>>>> 
>>>>> I am attaching 2 games as examples.
>>>>> 
>>>>> -Joseph
>>>>> <game1.fibs><game2.fibs>
>>>> 
>>