Re: [Bug-gnubg] Grade thresholds

[Joern Thyssen]

On Fri, Apr 04, 2003 at 10:35:22AM -0300, Albert Silver wrote
> I received another query on GNU's generosity in its grades
> (Supernatural, World Class, etc.) by a strong Brazilian player I
> introduced to GNU. He loves it, and has commented on how much faster it
> is to analyze matches or do rollouts with, but noted that it seemed more
> generous in its grades than Snowie 4. I can pass the e-mail to you, but
> should note it's in Portuguese. 
> 
> I'd like to pass on two possible requests therefore:
> 
> 1) just tighten the thresholds by a notch, so that Supernatural is at
> 0.04, WC at 0.08, etc. 

I would really like to see some statistical material on how the error
rate in Snowie relates to gnubg. In principle they should be identical
if the bot's played alike, but the matches I've seen they're very
different. 

If someone has a huge number of matches analysed with both Snowie and
gnubg I would really like to see a table with Snowie error rate vs gnubg
error rate. Perhaps 50 or so matches should do -- preferably between
human players.

>From this table we could calculate the approximate conversion factor,
and use it on the Snowie thresholds.

> 
> OR
> 
> 2) Allow users to customize the thresholds if they like, such as is
> possible for the move-by-move error thresholds.
> 
> One might argue that this last option would mean sharing results would
> give incompatible results, but frankly I don't spend my time passing
> results around nor do I believe others do, so I don't think it's much of
> an issue.

I really don't like that, because of the incompatible results.

Jørn