bug-gnubg
[Top][All Lists]

Re: [Bug-gnubg] An evalutaion of the pruning nets

 From: Robert-Jan Veldhuizen Subject: Re: [Bug-gnubg] An evalutaion of the pruning nets Date: Thu, 04 Nov 2004 16:04:20 +0100 User-agent: Mozilla Thunderbird 0.8 (Windows/20040913)

```Hi,

```
Some more about the pruning net results, I'll forward this from GOL (30 October):
```

***********************************************************************
I thought this was interesting:

```
Jim Segrave on the gnubg mailinglist, about the differences in a large sample (although SE are still very high) of matches analysed by both versions:
```
(...) The results:

..............................win.....wing...winbg...loseg...losebg cubeful

Average absolute difference 0.00123 0.00108 0.00009 0.00157 0.00020 0.00343

Std err.....................0.00619 0.00390 0.00051 0.00642 0.00403 0.02100

In 2,375 cases, the choice of best move differed, (0.94% of the time) (...)

[NOTE: Jim later changed this to 2.98% of the time]

```
I read some conclusions, probably based on this, that 2-ply prune is as good as 2-ply no prune, practically speaking. I'm paraphrasing here, sorry if I'm wrong about this.
```
```
However, looking at these figures I'm not so sure. In an absolute sense these differences look very small indeed. However, 2-ply is supposed to be playing a world-class (or better!) game. That means even the slightest increase in error rates means a clearly lower level of play.
```
```
I'm not sure how to quantify and interpret the figures, certainly not because the SE's are so high. But isn't it a bit early to draw conclusions from here that the pruning has "almost zero" effect on skill?
```
```
From some simulations I did myself, mostly letting gnubg play against itself, 0-ply vs. 2-ply and 2-ply reduced vs. 2-ply 100%, I think that the pruning net choosing a different move almost 1% of the time [NOTE: 3% even, it seems] is significant, again considering the fact that 2-ply is supposed to play world-class or better.
```*************************************************************************

```
It seems like "different move" included all positions where moves have equal equity so it doesn't matter. That makes the figures harder to interpret; part of the 3% differences is simply irrelevant. But if there's still 2% REAL differences, I think that's a significant difference, also looking at the equity differences Jim reports here (but which may not be of any value considering the SE?).
```
Greetings,
--
Robert-Jan Veldhuizen

```