Re: [gnugo-devel] regressions and bugs

[Gunnar Farnebäck]

Emanuele wrote:
> Hi people.
>
> First, thanks to everybody for this nice project.
>
> It's many years I use GNU Go and I will be happy
> to contribute a little.
>
> I know C language but my last acivity as professional
> programmer is dated 12 years ago, so I cant give much
> help in writing code now (not in a short time).
>
> Then I thougth I can start working on regressions.

You're welcome.

> Often using GNU Go I run up against "a bug" or,
> better, a bad move that is clearly wrong,
> especially in endgame and life and death problems.
>
> The question is: if i find a position in wich the
> program make a clear bad move, how can i contribute
> without (still) have the ability to correct the code?
>
> More in detail:
>
> a) Can I increment the regression test suite with
> sgf and tst files?

Yes.

> And if yes who will decide if the failed test is espected or not?

Usually we consider a failed test unexpected when it is added. 
Expectedness is updated for every development release (occasionally more 
often).

> Who has in charge the regression branch?

Nobody in particular but there's a limited number of people who have 
commit rights to the CVS repository (me, Dan, Arend, nando).

> b) Can I work on patterns without a deep study
> of the source code (owl and combination)?

Yes, it should in principle be possible but it surely helps to be able 
to read some of the involved code.

> c) The example in attach is useful or not?

Yes, it's useful but should be written somewhat differently.

> Wich criteria shuld I use to decide if a test
> is useful?

Most importantly it should be clear and unquestionable. We have no 
shortage of tests where someone has disliked some move for not at all 
obvious reasons, which makes it very difficult to analyze what's going 
on if a patch breaks or fixes a test. Comments about a test can help a 
lot to make it clear and unquestionable.

Second we try to avoid negative tests, where all but one or a few moves 
are accepted. These have turned out to cause "regression noise", 
typically because they can pass when another move is generated for a bad 
reason and then go back to failing when that bad reason has been fixed.

Third it's a plus, but not at all necessary, if the tests are 
systematic, e.g. like seki.tst and parts of ld_owl.tst. If you want to 
take on a bigger task in this area you can have a look at
http://trac.gnugo.org/gnugo/ticket/41

> loadsgf games/cisba/cisba01.sgf 223

I believe you mean 224 above, since that's white's move. Loadsgf loads 
up to but not including the given move number.

> 1 reg_genmove white
> #? [!H5]
> 2 reg_genmove white
> #? [J10|A1]

Number 1 is a negative test and should be avoided for that reason. 
Number 2 is not really interesting. To get the best effect of this 
position the test should use restricted_genmove, e.g.

1 restricted_genmove white H5 J4 H3 J3 H2 J2
#? [J4|H3|J3|H2|J2]

Also this is a special type of mistake, where the move is worse than 
passing, that we have collected in the test suite blunder.tst, so this 
particular test should be added at the end of that file.

/Gunnar