Re: [gnugo-devel] NNGS games in regression/games/nngs

[Trevor Morris]

At 07:12 PM 1/7/2002 -0800, you wrote:
>
>> >If the policy you propose is followed, we will pollute the regression
>> >tarball with a lot of games for which there are no tests.
>> I think that this will not be much of a problem, expecially compared 
>> to the benefit.  As I suggested before, the games without tests
>> should be weeded out every month or two (this would be very easy).
>> 
>> I suggest that we give the plan a try, and see if the pollution is
>> too bad.
>
>When a game is added to the regression directory and tuned against
>the amount of time spent adding it and giving it a name (a couple
>of minutes) is insignificant compared with the amount of time you
>spend making up tests for it and tuning against those tests.
>
>Tuning material is very ubiquitous. There is so much of it that
>indiscriminately adding every game or even a measurable percentage
>of all completed games is not feasible.
>
>GNU Go has played 8596 games as B and 9270 games as W on NNGS.
I'm only suggested new games, not old ones.

>Automatically adding these games to the regression tarball is not
>something we should be doing. Isn't this obvious?

In fact, it is far from obvious.  For example, I've pitted GNU Go 
against itself in 4 mini-tournaments (trevor[a-d].tst suites).  It 
took me no more than a couple of hours to create each test suite, which
are about 100 problems each.  That is, I can create a problem in about
1 minute, averaging 3-4 problems per game.  It is very easy for me to 
create a test suite, if the games are set up and easily accessible.

You see that I tend to spend only a couple of minutes per game, and
may generate 3-4 problems.  Therefore, downloading, and archiving
the games can take a significant portion of that time.

-Trevor