[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13
From: |
Albert Silver |
Subject: |
RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13 |
Date: |
Mon, 9 Jun 2003 17:01:59 -0300 |
First of all, many thanks for the clear explanation, and thanks for
doing the analysis in the first place.
> I've attached a file which has the following entries:
I saw no file BTW.
>
> game number, actual result, gnubg luck, snowie luck, luck adjusted
result
>
> For example, for game 100 (my example above):
>
> 100 1 -.01415 -.49717 .51698
>
> Some of the games could be very interesting to inspect carefully. For
> bots of similar strength we expect luck adjusted results around 50%.
> However, this is not always true in the 100 match sample you've sent
me:
>
> Examples:
>
> 8 0 .19603 .19853 .00250
> 39 1 .07856 .00087 .92231
>
> Either gnubg's luck analysis is totally wrong or snowie (gnubg)
> played very bad in game 39 (game 8).
Yes, there are problems with the analysis of some extreme games, and I
presume this also makes the luck analysis just as dubious. First, I'd
like to point out that I've noticed that GNU's evaluation (not
necessarily the play) of backgames at 2-ply can be (*can be*, not *is*)
extremely dubious at times and the odd-ply such as 3-ply will be very
close to reality. I have no idea why this is so. There was that position
from Dupreli's rollout comparison table, and below is another. I'm
sharing this example from match #79 game 6 as it shows a really wild
game in which GNU was extremely critical of a lot of Snowie's moves.
Considering the evaluation, I think it's probably mistaken.
Example (I laughed at this very extreme position BTW):
GNU Backgammon Position ID: CogAtnvYdhtQAA
Match ID : MIHxAGAAMAAA
+12-11-10--9--8--7-------6--5--4--3--2--1-+ O: Snowie4
| X O O O | | O O O X X X | 6 points
| O O O | | O O O X X X | Rolled 34
| O | | X |
| | | |
| | | |
^| |BAR| | 7 point match (Cube: 1)
| | | |
| | X | |
| | X | |
| | X | |
| X O O | X | X X | 6 points
+13-14-15-16-17-18------19-20-21-22-23-24-+ X: gnubg
1. Cubeful 2-ply 16/12* 7/4 Eq.: +0.657
82.8% 54.9% 35.6% - 17.2% 0.0% 0.0%
2-ply cubeful 100% speed [world class]
2. Cubeful 2-ply 9/5 7/4 Eq.: +0.607 (
-0.049)
80.4% 53.2% 34.3% - 19.6% 0.0% 0.0%
2-ply cubeful 100% speed [world class]
3. Cubeful 2-ply 8/4 7/4 Eq.: +0.607 (
-0.050)
80.4% 54.3% 35.7% - 19.6% 0.0% 0.0%
2-ply cubeful 100% speed [world class]
4. Cubeful 2-ply 16/12*/9 Eq.: +0.585 (
-0.071)
79.3% 50.7% 33.0% - 20.7% 0.0% 0.0%
2-ply cubeful 100% speed [world class]
* 13. Cubeful 2-ply 17/14 9/5 Eq.: +0.503 (
-0.154)
75.2% 48.4% 31.0% - 24.8% 0.0% 0.0%
2-ply cubeful 100% speed [world class]
This 2-ply evaluation is absurd needless to say. The 3-ply below is much
better (no idea about the move choices) though:
1. Cubeful 3-ply 16/12*/9 Eq.: +0.104
55.2% 35.7% 24.1% - 44.8% 0.0% 0.0%
3-ply cubeful [grandmaster]
2. Cubeful 3-ply 17/14 16/12* Eq.: +0.096 (
-0.007)
54.8% 36.4% 23.4% - 45.2% 0.1% 0.0%
3-ply cubeful [grandmaster]
3. Cubeful 3-ply 16/12* 7/4 Eq.: +0.075 (
-0.029)
53.8% 34.0% 23.4% - 46.2% 0.0% 0.0%
3-ply cubeful [grandmaster]
4. Cubeful 3-ply 17/13 8/5 Eq.: +0.072 (
-0.032)
53.6% 34.8% 23.4% - 46.4% 0.1% 0.0%
3-ply cubeful [grandmaster]
9. Cubeful 3-ply 17/14 9/5 Eq.: +0.048 (
-0.056)
52.4% 33.5% 23.2% - 47.6% 0.0% 0.0%
3-ply cubeful [grandmaster]
Albert
>
> Match 39 re-analysed:
>
> 0-ply: 1 .07856 .00087 .92231
> 1-ply: 1 .1150 -.0151 .87449
> 2-ply: painfully slow; I gave up
>
> The result is changed by 5%, but we're still far from a luck adjusted
> result of 50%. I can't explain this...
>
> Jørn
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, (continued)
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Rod Roark, 2003/06/09
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Joern Thyssen, 2003/06/09
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Rod Roark, 2003/06/09
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Joern Thyssen, 2003/06/09
- RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Albert Silver, 2003/06/09
- RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Albert Silver, 2003/06/09
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Joern Thyssen, 2003/06/09
- RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13,
Albert Silver <=
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Joern Thyssen, 2003/06/09
- Re: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Joseph Heled, 2003/06/09
- RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Albert Silver, 2003/06/09
RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Ian Shaw, 2003/06/09
RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Ian Shaw, 2003/06/09
RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Ian Shaw, 2003/06/09
RE: [Bug-gnubg] Snowie 4 vs. GNU 0.13, Joseph Heled, 2003/06/10