Re: [Bug-gnubg] Gap in guestimated ratings?

[Joern Thyssen]

On Sun, Jan 26, 2003 at 02:00:41AM -0500, James F. Kibler wrote
> Joern Thyssen wrote:
> > 
> > The error rate per decision can easily exceed 1 (for example, if you
> > cube decisions are really lousy).
> 
> So, I guess this means that the 'error rate per decision' is really
> 'error rate per move'?  Do you count an error for each checker movement
> and an error in the cube decision in the calculation, yielding a maximum
> of 3 errors per move?

The error rate per decision is calculated as the total error (in
normalised equity) for chequer play decision plus the total error for
cube decisions divided by the number total number of decisions:

Total moves                     115                     116
Unforced moves                  109                     106
Error rate (total)              -1.583 (-35.923%)       -2.927 (-40.369%)
Error rate (per move)           -0.015 ( -0.330%)       -0.028 ( -0.381%)

Total cube decisions             59                      77
Close or actual cube decisions   36                      19
Error rate (total)              -1.357 (-16.595%)       -0.295 ( -3.723%)
Error rate (per cube decision)  -0.038 ( -0.461%)       -0.016 ( -0.196%)

Error rate (total)              -2.940 (-52.517%)       -3.222 (-44.091%)
Error rate (per decision)       -0.020 ( -0.362%)       -0.026 ( -0.353%)

Guestimated abs. rating         1741.64                 1584.42


The error rate per move for player 0 is: -1.583/109 = -0.145, and
-1.357/36 = -0.0377 for cube decisions.

The total error rate is (-1.583 + -1.357) / ( 109 + 36 ) = -0.020, which
is fed into my interpolation formula to yield a rating of 1741
(linear interpolation between (1600,-0.025) and (1750,-0.02).

> > > What is the second argument (n) to the rating function?
> > 
> > The match length. The current interpolation scheme does not use it.
> 
> If the rating were a function of match length, any idea what the
> relationship might be?  Guess I should start tracking match length as
> well.  

I don't know, but the FIBS rating system "overrates" 1 point matches, so
the match length may have to go into the interpolation formula in some
form.

> I currently have about 500 FIBS match logs.  Is there a batch procedure
> to allow gnubg to analyze all of them at one time, hopefully producing a
> text file as output from each match analysis?

>From the mail headers I gather you run windows 95. There used to be a
gnubg-no-gui.exe which is very useful for this purpose.

You can write a DOS batch file like the following unix shell script:

#!/bin/sh
for ff in *.fibs; do

    $sgf=`echo $ff | sed -e 's/\.fibs/.sgf/g'

    gnubg t < EOF >> run.log
set priority idle
import fibs $ff
analyse match
save match $sgf
show score
show stat match
EOF

done

Maybe someone can help re-writing this into a DOS batch file?

>From the produced file run.log we can extract the match length and the
error rate per decision.

Jørn