Re: [Bug-gnubg] Strange analysis result

[Jim Segrave]

On Thu 06 Feb 2003 (00:12 +0200), Lauri Salo wrote:
> Hi,

[reformatted to reasonable line lengths]
 
> I just analysed my match with GNU and Snowie and the results were
  strange. Snowie rated me as expert (nearly world class) and my
  opponent as advanced, with average error rates 4.4 and 7.7
  respectively. Checker play and doubling-wise it went: EU_FIN_Vrabec
  (checker play: 10 errors, no blunders; doubling: 2 errors, 1
  blunder), StileBlue (checker play: 9 errors including 5 blunders;
  doubling: 2 errors, no blunders). So according to Snowie my checker
  play was superior and StileBlue's doubling play slightly better than
  mine.
 
> GNU evaluated doubling play similarly, but gave a strange evaluation on 
> checker play:
> 
> Player                      StileBlue                      EU_FIN_Vrabec  
> 
>  
> 
> Checkerplay statistics:
> 
>  
> 
> Total moves:                       99                              98
> 
> Unforced moves:                    71                              68
> 
> Moves marked very good              0                               0
> Moves marked good                   0                               0
> Moves marked interesting            0                               0
> Moves unmarked                     91                              90
> Moves marked doubtful               1                               6
> Moves marked bad                    5                               1
> Moves marked very bad               2                               1
> 
> Error rate (tot       +1.152 (+12.137%)              +2.468 (+106.991%)
> 
> Error rate (pr. move) +0.016 ( +0.171%)              +0.036 ( +1.573%)
> 
> Checker play rating:           Advanced                       Beginner 
 
> Looking at the "moves marked doubtful/bad/very bad" GNU's analysis
> seems to agree with Snowie, with slight differences, but
> contradicting itself. I went through the errors I made and I'm
> positive that the "very bad move" I made couldn't possibly be so bad
> that it was a complete "match destroyer" as the analysis seems to
> suggest. I appreciate that the logic between the two bots is
> different and I often get slightly different analysis when I compare
> the two, but it's the first time something is this much off the
> mark. Any idea what happened?  

It's hard to guess without seeing the match and the play(s) in
question. If you were to save the match as a .sgf file
(File->Match->Save) then there are several good players who may be
able to answer if it looks like a really serious error. The other
thing you can do is use a rollout to see if the equity a rollout gives
for your move and the move gnubg or Snowie considers the best move. It
is possible that one or the other or both programs make mistakes in
evaluating some positions, and can give a seriously wrong answer. It
can also depend on the settings you are using for analysis.


-- 
Jim Segrave           address@hidden