|
From: | Joseph Heled |
Subject: | Re: [Bug-gnubg] Rating and grading |
Date: | Tue, 08 Jul 2003 08:21:36 +1200 |
User-agent: | Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624 |
Rod Roark wrote:
On Monday 07 July 2003 10:40 am, Albert Silver wrote:... GNU's attributed rating is supposedly based on the FIBS rating system and is designed to give a rating that approximates what a user might get at FIBS. However, it's self-attributed rating seems to be what everything else is relative to and is at 2200. GNU does not have a 2200 rating at FIBS, not even by a margin. I think that lowering this to 2050, representing it's average, would make much more sense. As a consequence, all its relative ratings should be lowered accordingly.Here are the latest cumulative stats for ParlorBot, which runs gnubg on FIBS with 2-ply checker play and cube decisions, 33% reduced:
I have found that 33% reduced for checker play is not really good. Using small pruning nets give you the speedup with a much reduced error rate.
Number of samples : 3692 (i.e. number of rated matches played) Lowest : 1880.71 Highest : 2120.74 Mean : 2002.35 Standard Deviation : 51.70 While the average rating used to be around 2030, there has been a great deal of wandering from that. Also I suspect that a few clever FIBSters have learned to take advantage of the bot's less-than-perfect evaluation in back games and holding games.
The FIBS rating formula is not perfect. In my small tests (using 2 different nets as players) the weaker player has an advantage, i.e. wins more of his 3,7,9,... matches than what is implied by the 1ptr win rate. So, if you stuck mainly playing against weak players, your rating drops. If you play stronger ones, it goes up.
We need lots of data - i.e. matches between GNUbg and players - analyzed their error rate - and try to come up with a formula that better matches the FIBS number.
-Joseph
-- Rod _______________________________________________ Bug-gnubg mailing list address@hidden http://mail.gnu.org/mailman/listinfo/bug-gnubg
[Prev in Thread] | Current Thread | [Next in Thread] |