Re: [gnugo-devel] 9x9 opening stats against humans

[Gunnar Farnebäck]

Doug wrote:
> Since self play doesn't seem like a good way to test move quality, I
> took a look at play against people. I downloaded all May 2004 games
> from all the KGS bots running various versions of GnuGo [1], and
> selected 9x9 games with no handicap and komi between 5.5 and 8.0. I
> did no selection based on opponent. I looked at win frequency by
> opening pattern, and also looked for opening patterns played by pros
> [2] which were not played at all.

Interesting analysis.

>   * Pros do indeed play moves that the amateurs in the original training
> set played rarely or never.

How big set of pro games do you have? (Just curious, it does of course
not matter for this conclusion.)

>   * It will take a lot of data to prove mediocre moves as definitively
> worse.

No question about that.

>   * The move weighting, based on popularity in the original training set,
> seems problematic. First, since a deweighted move is played more rarely,
> it takes more games to figure out if it's better or worse than the higher
> rated moves.

That's problematic as far as evaluation goes, but I don't think it's a
direct playing strength problem.

> Second, the more moves are weighted, the more predictable they
> become.

But the higher weighted moves let us stay in the fuseki database
longer, usually reducing the predictability.

> And finally, it looks like some of the best moves are played
> fairly rarely, so changing the weights could improve average play.

That may very well be possible. 

> I hope to figure out how to add the pro game patterns to the existing set.

It's probably easier and/or better to recompile the database from
scratch, assuming you have or can get a large enough training set. You
might want to patch extract_fuseki.c to give higher weight to the pro
games.

> Would anyone mind if I changed the weights at the same time?

Probably not, although it might depend on by what principles you
intend to modify them. :-)

> Also, does anyone have the original human-human 9x9 training set
> around?

The current fuseki databases were created by MÃ¥ns Ullerstam three
years ago. He is no longer on the list but you can try to mail him
privately to see if he still has the games around.

/Gunnar