[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-gnubg] Rating and grading

From: Joseph Heled
Subject: Re: [Bug-gnubg] Rating and grading
Date: Tue, 08 Jul 2003 08:21:36 +1200
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624

Rod Roark wrote:
On Monday 07 July 2003 10:40 am, Albert Silver wrote:

... GNU's attributed rating is supposedly based on the FIBS rating system
and is designed to give a rating that approximates what a user might get
at FIBS. However, it's self-attributed rating seems to be what
everything else is relative to and is at 2200. GNU does not have a 2200
rating at FIBS, not even by a margin. I think that lowering this to
2050, representing it's average, would make much more sense. As a
consequence, all its relative ratings should be lowered accordingly.

Here are the latest cumulative stats for ParlorBot, which
runs gnubg on FIBS with 2-ply checker play and cube
decisions, 33% reduced:

I have found that 33% reduced for checker play is not really good. Using small pruning nets give you the speedup with a much reduced error rate.

Number of samples  : 3692   (i.e. number of rated matches played)
Lowest             : 1880.71
Highest            : 2120.74
Mean               : 2002.35
Standard Deviation :   51.70

While the average rating used to be around 2030, there has
been a great deal of wandering from that.  Also I suspect
that a few clever FIBSters have learned to take advantage of
the bot's less-than-perfect evaluation in back games and
holding games.

The FIBS rating formula is not perfect. In my small tests (using 2 different nets as players) the weaker player has an advantage, i.e. wins more of his 3,7,9,... matches than what is implied by the 1ptr win rate. So, if you stuck mainly playing against weak players, your rating drops. If you play stronger ones, it goes up.

We need lots of data - i.e. matches between GNUbg and players - analyzed their error rate - and try to come up with a formula that better matches the FIBS number.


-- Rod

Bug-gnubg mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]