Re: [Bug-gnubg] Is gnubg's game-analyisis function always reliable?

[Neil Robins]

I suspect the answer is in your use of 3-ply for cube decisions. It may be
called "grandmaster" but in 90% of the rollouts I have done 2-ply is closer
to the result of the rollout. In some positions 3-ply is better, but I've
also seen several positions that it thinks are too good to double that roll
out as no double/take. If you are doubling too early and dropping too easily
you won't be seeing most of your cube errors because those are the mistakes
that 3-ply cube analysis makes too. I would strongly recommend any even play
cube analysis over it.

 
Adi Kadmon wrote:
> 
> Dear Developers
> 
> 
> 
> I have Version GNU Backgammon 0.90-mingw 20090403, and I’ve been training
> a
> lot with it and finding it very strong.
> 
> 
> 
> However, lately I’ve been encountering a phenomenon for which I cannot
> account. Analyzing the money-games (a single game each time) with some of
> my
> opponents using the feature “Analysis” (set to “supremo” 2-ply for checker
> play and to “grandmaster” 3-ply for cube decisions), I found consistently
> for some of the opponents that I was significantly better than they were
> by
> the analysis` measures, and nevertheless I consistently lost to them more
> than gain, sometimes in a ratio approaching 2:1 or 3:2!
> 
> 
> 
> For instance, Analyzing 69(!) games with a certain opponent, with the
> above
> settings for the analysis function, I found the following:
> 
> 
> 
> In 41 games I had an error rate mEMG per move (total: checker play + cube
> decisions) lower than his. The advantage was his only in 27, and in 1 game
> we came out equals.
> 
> 
> 
> In 43 games I had an error rate mEMG per move (checker play) lower than
> his,
> whereas he had the advantage in 26 games only.
> 
> 
> 
> In 29 games I had an error rate mEMG per cube decision lower than his,
> whereas he had the advantage in 21 only, and in 19 games we came out
> equals.
> 
> 
> 
> Seeing all this, it was all the more disconcerting to find that on average
> I
> lost 0.2826 points per game during these 69 games!
> 
> 
> 
> Note: the “marks” (“world-class…… awful”) for me and for him were
> consistent
> with the above analysis measures. The bot found my opponent to be luckier
> than me on average (unfortunately I didn’t keep the exact numbers of games
> on his side or mine) – but, for heaven’s sake, such luck through 69
> game!!!
> With such an ostensible advantage on my side? It’s highly improbable. And
> this opponent is, as I said, not the only example.
> 
> 
> 
> I’ve been trying to imagine an explanation for these facts, such as:
> 
> 
> 
> (1)   That opponents, and a few more, happened to be definitely stronger
> than gnubg. But stronger than “supremo” and “grandmaster” in such a
> decided
> and consistent fashion – is it conceivable?
> 
> (2)   Those opponents somehow cheated heavily on the dice (on Play65 site)
> –
> but I deem it very unlikely, and I’m not used to being paranoid at all…
> 
> (3)   My opponents’ style was different from gnubg’s, to which I’m much
> more
> accustomed (and from which I indeed learned much), even though
> theoretically
> their moves are a bit inferior by its standards – and one has to “know how
> to win against their style” in order to succeed against them. But is such
> a
> thing likely? I would have rather thought that if by a very strong bot’s
> standards my moves are superior on average, it’s not a relativistic but a
> definite superiority and should beat a “different style”.
> 
> (4)   My errors occur in such early (or late) stages in the game in
> comparison with the opponent’s errors, that my errors are “more costly”,
> more decisive for the final result. However, I doubt this too: Wouldn’t
> the
> bot than have found my mistakes statistically very grave for the relevant
> positions, thereby affecting the above measures accordingly so that on
> average they again should turn out inferior, not superior?!
> 
> 
> 
> 
> 
> Well, I trust your vast knowledge and experience would yield a suitable
> explanation, or at least point to some possible explanations.
> 
> 
> 
> Please do help!
> 
> 
> 
> Sincerely,
> 
> 
> 
> Adi
> 
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/bug-gnubg
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Is-gnubg%27s-game-analyisis-function-always-reliable--tp23233807p23391894.html
Sent from the Gnu - Backgammon mailing list archive at Nabble.com.