Re: [Bug-gnubg] Confused

[Michael Petch]

On 2015-06-12 8:24 AM, Ian Shaw wrote:
> 
> You make a good point about Michael's benchmark. Are you repeating these 
> tests? If not, if somebody has the positions in a file, I might be able to 
> devote some CPU time to it. 
> 
> The tests I ran were several years ago, so they weren't with the latest 
> weights. I think they'd have been the 0.14 weights.
> 
> Did Michael give you evidence that 3-ply was worse than 2-ply? I don’t 
> remember any tests being reported on the mailing list, and I'd be very 
> interested.
> 

I don't think Michael Depreli drew any conclusions but I believe others
(as quoted at the bottom of the study reinforced the idea that 3ply for
cube decisions was a poor choice). The study is here:
http://www.bkgm.com/articles/Keith/DepreliBotComparison/

I told Lucas about the odd/even ply effect and directed him to the
Depreli study. That study was done with the Pre-1.00 neural nets that
(The Rudman DLL is well over a decade old and definitely uses the older
net). I don't recall saying "a little". There is no conclusive evidence
since I don't recall a full study being done, but Philippe Michel did
ask me (around the time of the 1.00 release) to run the new neural net
through the Depreli positions to see how the new net faired. The results
of that can be found here:

http://www.capp-sysware.com/downloads/gnubg/gnubg_prelim_results-1_00.txt

If you review Grandmaster and the experimental 2ply chequer/3ply cube
with the old and new net relative to 2ply and 4ply you will see that it
did does quite a bit better. Whether the Depreli study translates to
general play and positions is not known. If someone wishes to do such a
study then all the power to them (I haven't discouraged anyone from
doing such an experiment)

The numbers in my chart can be used as a reasonable comparison to the
other data in Depreli's "Bot Comparison — 2012 Update" which can be
found in the first link I gave.

-- 
Michael Petch
GNU Backgammon Maintainer / Developer
OpenPGP FingerPrint=D81C 6A0D 987E 7DA5 3219 6715 466A 2ACE 5CAE 3304