[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-gnubg] An evalutaion of the pruning nets

From: Robert-Jan Veldhuizen
Subject: Re: [Bug-gnubg] An evalutaion of the pruning nets
Date: Thu, 04 Nov 2004 16:04:20 +0100
User-agent: Mozilla Thunderbird 0.8 (Windows/20040913)


Some more about the pruning net results, I'll forward this from GOL (30 October):

I thought this was interesting:

Jim Segrave on the gnubg mailinglist, about the differences in a large sample (although SE are still very high) of matches analysed by both versions:

(...) The results:

..............................win.....wing...winbg...loseg...losebg cubeful

Average absolute difference 0.00123 0.00108 0.00009 0.00157 0.00020 0.00343

Std err.....................0.00619 0.00390 0.00051 0.00642 0.00403 0.02100

In 2,375 cases, the choice of best move differed, (0.94% of the time) (...)

[NOTE: Jim later changed this to 2.98% of the time]

I read some conclusions, probably based on this, that 2-ply prune is as good as 2-ply no prune, practically speaking. I'm paraphrasing here, sorry if I'm wrong about this.

However, looking at these figures I'm not so sure. In an absolute sense these differences look very small indeed. However, 2-ply is supposed to be playing a world-class (or better!) game. That means even the slightest increase in error rates means a clearly lower level of play.

I'm not sure how to quantify and interpret the figures, certainly not because the SE's are so high. But isn't it a bit early to draw conclusions from here that the pruning has "almost zero" effect on skill?

From some simulations I did myself, mostly letting gnubg play against itself, 0-ply vs. 2-ply and 2-ply reduced vs. 2-ply 100%, I think that the pruning net choosing a different move almost 1% of the time [NOTE: 3% even, it seems] is significant, again considering the fact that 2-ply is supposed to play world-class or better.

It seems like "different move" included all positions where moves have equal equity so it doesn't matter. That makes the figures harder to interpret; part of the 3% differences is simply irrelevant. But if there's still 2% REAL differences, I think that's a significant difference, also looking at the equity differences Jim reports here (but which may not be of any value considering the SE?).

Robert-Jan Veldhuizen

reply via email to

[Prev in Thread] Current Thread [Next in Thread]