Re: [Bug-gnubg] Rollout with 'Stop when result is accurate'

[Philippe Michel]

On Sat, 31 Oct 2009, Yoshito Takeuchi wrote:

> I rollout with 'Stop when result is accurate'.
> But, Standard error enough under set value, non stop GnuBG rollout.
> I use latest version GnuBG ( Today I compile gnubg from source code).
>
> My setting.
>
> Settings-rollouts
>
> set 'Trials:' 12960
> Check 'Stop when STDs are small enough'
> set 'Minimum Trials:' 648
> set 'Ratio|Standard Deviation/Value:' 0.0200

'Stop when STDs are small enough' may not do what you seem to be 
expecting. As stated, it stops when Standard Deviation/Value (not equity 
sd) is below the limit you set *for every output* : singles wins, gammons, 
backgammons for both players, cubeless and cubeful equity.

In practice, this criterion is useless : backgammon numbers are often very 
noisy, if the equity is close to 0, sd/equity is high, etc...

Since you compile from source, you may want to apply the appended patch 
that changes the criterion to 'stop when equity sd is small enough'.

With it, I get results like :

Rollout details:
Centered 1-cube:
  0.71334 0.02413 0.00064 - 0.28666 0.02212 0.00057 CL  +0.42875 CF 
+0.72226
 [0.00030 0.00028 0.00006 - 0.00030 0.00025 0.00004 CL   0.00070 CF 
0.00175]
Player gnubg owns 2-cube:
  0.71590 0.02384 0.00063 - 0.28410 0.02205 0.00057 CL  +0.86731 CF 
+0.71301
 [0.00031 0.00029 0.00006 - 0.00031 0.00026 0.00004 CL   0.00141 CF 
0.00200]
Full cubeful rollout with var.redn.
12051 games, Mersenne Twister dice gen. with seed 868626794 and 
quasi-random dice
Stop when std.errs. are small enough: ratio 0.002 (min. 1296 games)

which may be what you want. With stock gnubg, it wouldn't have stopped 
there since for instance sd(bg)/bg for the player on roll is about 0.1, 50 
times too high!

Index: rollout.c
===================================================================
RCS file: /sources/gnubg/gnubg/rollout.c,v
retrieving revision 1.221
diff -u -r1.221 rollout.c
--- rollout.c   1 Oct 2009 21:05:54 -0000       1.221
+++ rollout.c   31 Oct 2009 19:20:15 -0000
@@ -1216,7 +1216,7 @@
                                if (fNoMore[alt] || altGameCount[alt] < 
(rcRollout.nMinimumGames ))
                                        continue;
                                prc = &ro_apes[alt]->rc;
-                               for (output = 0; output < NUM_ROLLOUT_OUTPUTS; 
output++) {
+                               for (output = OUTPUT_EQUITY; output < 
NUM_ROLLOUT_OUTPUTS; output++) {
                                        if (output < OUTPUT_EQUITY) {
                                                v = fabs(aarMu[alt][output]);
                                                s = fabs(aarSigma[alt][output]);
@@ -1249,8 +1249,7 @@
                                                                           
&aciLocal[(ro_fCubeRollout ? 0 : alt)]));
                                                }
                                        }
-
-                                       if ((v >= .0001) && (v * 
rcRollout.rStdLimit < s)) {
+                                       if ( rcRollout.rStdLimit < s) {
                                                err_too_big = 1;
                                                break;
                                        }

PS: Jon, Christian, how about applying this change (or something to that 
effect), plus the corresponding update the UI ? I insist : the current 
behaviour of this criterion is just useless!