Re: [gnugo-devel] opening db proposal

[Paul Pogonyshev]

Douglas Ridgway wrote:
> On Thu, 21 Oct 2004, Paul Pogonyshev wrote:
> 
> > Douglas Ridgway wrote:
> > > 
> > > [...]
> > > 
> > > I haven't tried to restrict total number of patterns for its own sake.
> > > This might be an issue in the 19x19 even db. If so, we need to decide
> > > how big it can be.
> > 
> > How large is the increase in the binary size for your updated fuseki
> > database?
> 
> On my machine, a clean cvs gnugo binary takes 4.5M on disk, and compiles
> in about 65 s. With the existing fuseki19.db replaced by just the big even
> db, no handicap patterns, gnugo occupies 7.9 M on disk and compiles in
> about 105 s. I haven't built one with all the new patterns, but I'd guess
> the total damage would be about a factor of two in both binary size and
> compile time. I don't see an increase in memory footprint, but I'm 
> probably just doing something wrong.

This is quite bad.  One simple measure is to pack fullboard patterns with
`patval_compact' entries instead of `patval', where `patval_compact' can
be defined as

  struct patval_compact {
    short offset;
    char  att;
  }

This alone packs fuseki database almost two times.

Another possibility is to try to enhance corner matcher to handle
full-board patterns, though I have to admit I cannot guarantee it will give
an improvement.

Concerning the compilation time, I think nothing can be done about it.  From
what I have seen so far, it's the C compiler eating most of the time, not
`mkpat'.  In any case, unless one performs `make clean' often, this is not
a serious issue.

> > It would be good to test for especially dumb patterns before using this
> > database in upstream versions.  May be a non-trivial task given 20,000+
> > patterns though.
> 
> To a certain extent, this ought to be handled by the strength and
> popularity screening: any move which gets played repeatedly by dan level
> humans ought not be too bad. Still, it makes sense to look, not least
> because the program/training set may be buggy. Here's a couple which don't
> look reasonable to me (post 0x13c9b807 and 0x02e29b74), found by looking
> for moves with differences in the training set:
> 
> +-------------------+
> |...................|
> |...................|
> |...............O...|
> |...X...............|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |..*................|
> |...X...........O...|
> |...................|
> |...................|
> |...................|
> +-------------------+
> 
> 
> 
> +-------------------+
> |...................|
> |...................|
> |...*...............|
> |...X.....X.....X...|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...................|
> |...O...........O...|
> |...................|
> |...................|
> |...................|
> +-------------------+
> 
> 
> Unfortunately, both of those moves did substantially _better_ than the
> alternatives. Since all the players in this set are far stronger than me,
> I'm not really qualified to comment.

Both these are non-standard.  Kogo's joseki dictionary doesn't even list
the diagonal attachment.  I'd suggest to comment these out, if only because
GNU Go will never know how to follow them up.

Paul