[Bug-gnubg] Snowie error rates versus gnubg error rates

[Joern Thyssen]

Hi,

I've studied snowie error rates versus gnubg error rates a bit:

m=number of chequer play decisions (m'=number of unforced moves)
c=number of cube decisions         (c'=number of non-trivial cube decisions)

Em=total normalised error for chequerplay decisions
Ec=total normalised error for cube decisions


Snowie error rates:

Chequerplay error rate:  Em/(m+c)
Cube error rate:         Ec/(m+c)
Total error rate:        (Ec+Em)/(m+c)

gnubg error rates:

Chequerplay error rate:  Em/m'
Cube error rate:         Ec/c'
Total error rate         (Ec+Em)/(m'+c')


So gnubg's error rates will in general be much higher since we for all
three numbers divide by smaller numbers.

Consider the statement "My snowie chequerplay error rate was 4.0" versus
"My gnubg chequerplay error rate was 4.0".  The latter says that for
each unforced chequerplay decision I made an average error of 4.0
millipoints, whereas the former says that my chequerplay errors averages
to 4.0 millipoints per decision (chequerplay and cube dcisions
combined).

With a slight bias I prefer gnubg's definition. However, we're back to
the 0-ply vs. 1-ply discussion: do we match snowie's definition, or do
we think it's so lousy that we stick with our own!!!

In any case, it's fairly easy to add the equivalent Snowie error rate in
the output, so we should decide whether to:

(a) just display the gnubg error rate
(b) display both the gnubg and snowie error rate
(c) display the snowie error rate only

ad (a): I've recieved a sample of 50 matches from Albert. I can use them
to deduce an empirical conversion factor between gnubg and snowie error
rates. This can be used to adjust the threshold for assigning the
ratings "supernatural", "world class" etc, so gnubg and snowie are more
aligned.

So what's your opinion?

Jørn