Re: [Bug-gnubg] pubeval benchmark

[Joseph Heled]

Thanks for all of you who chipped in!

Playing a 1 point match at 0 ply, gnubg won 7193 of 10000 matches,
which is 0.4386 ppg in my book.
Playing 10000 money sessions (gammons count, no cube) gnubg got 0.6296
ppg (the breakdown is 4504, 2298, 285   2202, 676, 35 )

-Joseph

On 20 January 2012 06:16, Nikos Papahristou <address@hidden> wrote:
> My evaluations of a starting roll 3-1 are exactly the same with Mark.
> Joseph, my implementation of pubeval doesn't agree with the moves
> selection in the games you sent.
> In the first game, the first roll of pubeval (4-5) is played as 24/15,
> and in the second game the first roll (52) is played as 13/8, 13/11.
>
> Nikos
>
> On 19/1/2012 17:14 μμ, Mark Higgins wrote:
>> I get something different for a 3-1 off the starting board. Sorry for
>> the non-standard format but hopefully it's intelligible. I'm printing
>> out the layout and the pubeval value, ordered by descending value
>> (based on my pubeval implementation). 8-5 6-5 is the best move.
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o o| o *|*
>> *|* o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 10.3431*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 7.22287*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| |*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o| o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 6.73507*
>>
>> *|| | | | | ||| | | | | ||*
>> *| o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 6.64798*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| |*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o| o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 6.61369*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 6.44695*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o| o *|*
>> *|* o|o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 6.17616*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| |*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o o| o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 6.09006*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| |*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o| o *|*
>> *|* o|o o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 5.81926*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o| o *|*
>> *|* o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 5.76592*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| | *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 5.65459*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o| o *|*
>> *|* o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 5.63356*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o o *| * o|*
>> *| *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 5.55805*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| o| *|*
>> *| o| o *|*
>> *|* o| o *|*
>> *|* o o| o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 5.44036*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| | *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o| o *|*
>> *|* o o|o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 4.76362*
>>
>> *|| | | | | ||| | | | | ||*
>> *|o *| * o|*
>> *|o *| * o|*
>> *| *| * o|*
>> *| *| o|*
>> *| *| o|*
>> *| | |*
>> *| | |*
>> *| | |*
>> *| o| *|*
>> *| o| *|*
>> *| o| *|*
>> *|* o| *|*
>> *|* o o|o o *|*
>> *|| | | | | ||| | | | | ||*
>> *Value 3.78692*
>>
>>
>> On Jan 19, 2012, at 9:57 AM, Joseph Heled wrote:
>>
>>> Here are all the possibilities for a 3,1 from the initial position. I
>>> am worried that 8-5 6-5 sneaks at number 6
>>>
>>> score gnubg id position
>>> 8.00384996831 4HPwATDgc/ABIQ (0, 1, 0, 0, 0, 1, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0)
>>> 7.83777993917 4HPwATDEc/ABKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 3, 0, 4, 0, 0, 1, 0, -2, 0)
>>> 7.81063996255 4HPwATDgc/ABEg (0, 0, 1, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0)
>>> 7.75730997324 4HPwATDgc+IBKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
>>> -5, 0, 1, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0)
>>> 7.43968993425 4HPwATDQZ/ABKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 2, 0, 5, 1, 0, 0, 0, -2, 0)
>>> 6.84054994583 4HPwATCwZ/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 2, 0, 4, 2, 0, 0, 0, -2, 0)
>>> 6.81688992679 4HPwATDga/ABIg (0, 1, 0, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 2, 1, 5, 0, 0, 0, 0, -2, 0)
>>> 6.67967991531 4HPwATDQc/ABIg (0, 1, 0, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 3, 0, 4, 1, 0, 0, 0, -2, 0)
>>> 6.67962995172 4HPwATDCc/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 3, 0, 4, 0, 0, 0, 1, -2, 0)
>>> 6.24834996462 4HPwATDIZ/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 2, 0, 5, 0, 1, 0, 0, -2, 0)
>>> 6.10885995626 4HPwATDgc+EBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
>>> -5, 0, 0, 1, 3, 0, 5, 0, 0, 0, 0, -2, 0)
>>> 4.3719098568 4HPwATDEa/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 2, 1, 4, 0, 0, 1, 0, -2, 0)
>>> 4.29143989086 4HPwATDga+IBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
>>> -5, 0, 1, 0, 2, 1, 5, 0, 0, 0, 0, -2, 0)
>>> 4.23469984531 4HPwATCkc/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 3, 0, 3, 1, 0, 1, 0, -2, 0)
>>> 4.15422987938 4HPwATDQc+IBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4,
>>> -5, 0, 1, 0, 3, 0, 4, 1, 0, 0, 0, -2, 0)
>>> 3.22565984726 4HPwATDQV/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5,
>>> -5, 0, 0, 0, 1, 1, 5, 1, 0, 0, 0, -2, 0)
>>>
>>>
>>> On 20 January 2012 03:45, Mark Higgins <address@hidden
>>> <mailto:address@hidden>> wrote:
>>>> :) I'll give it a try and check the couple games you sent.
>>>>
>>>>
>>>> On Jan 19, 2012, at 9:39 AM, Joseph Heled wrote:
>>>>
>>>>> I was asking for help, not for more work.
>>>>>
>>>>> For example, anyone who implemented play against pubeval can check a
>>>>> few of the moves I sent.
>>>>>
>>>>> -Joseph
>>>>>
>>>>> On 20 January 2012 03:35, Mark Higgins <address@hidden
>>>>> <mailto:address@hidden>> wrote:
>>>>>> I used the following post to define pubeval; is this the proper
>>>>>> definition still:
>>>>>>
>>>>>> http://www.bkgm.com/rgb/rgb.cgi?view+610
>>>>>>
>>>>>> ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jan 19, 2012, at 8:31 AM, Joseph Heled wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I just run 10000 1 point match games between gnubg and pubeval. I
>>>>>>> think this is the most fair, since there are no gammons of cubes
>>>>>>> involved.
>>>>>>>
>>>>>>> However, gnubg won 9840 of those, which makes me think something
>>>>>>> is wrong.
>>>>>>>
>>>>>>> Can someone help me verify that pubeval makes the "right" moves?
>>>>>>>
>>>>>>> I can send a few games in oldmoves format, or can check specific
>>>>>>> moves.
>>>>>>>
>>>>>>> I am attaching 2 games as examples.
>>>>>>>
>>>>>>> -Joseph
>>>>>>> <game1.fibs><game2.fibs>
>>>>>>
>>>>
>>
>