Re: [gnugo-devel] Patch: improve separation of similarly-valued moves

[Arend Bayer]

I assume we agree from watching games against on NNGS that lack of
indeterminacy is a practical and not a theoretical problem. (You may
also recall that it was one of the complaints on GnuGo in the Go++ vs 
GnuGo comparison in a German games magazine for which I have an English
summary.)

> I don't think it would be too hard to create regression tests of the
> form "make sure all of the following moves will get played at least
> occasionally from this position," though clearly it would take a new GTP
> command.  I might give that a try at some point, though I don't know
> when I'll get to it.  If anyone wants to, they're more than welcome to
> do it ;)  I think that adding a test framework to test our ideas about
> keeping variation at different points in the game would help.

While some regression tests might be useful (i.e. specifically testing
that we do not regress in situation that we have tuned for), I think
even better would be an indetermincay benchmark.

The base would be a GTP command move_propabilities that would give
the percentages with which each of the top moves would get played
(assuming a random random seed). 100% C4 gives no indeterminacy,
50% C4, 50% D4 would give 1 bit indeterminacy (I think the value of
information of the gg_rand()-influenced decision is the right metric here)
etc., and then we just add up these interminacy values over the first 50
or 100 moves of a couple of games.

Then e.g. we could take Inge's patch, increase the randomness from 0.01
to 0.02 and say "Hey, 15 PASSes 5 FAILs and still a net gain in
indeterminacy!".

I actually haven't thought about the problem how hard it would be to 
compute the exact move probabilities in our current scheme, but it
should be doable. If nothing else, a Monte Carlo-approach would work
well enough (i.e. simulating the decision for 1000 gg_rand()s).

Arend