Re: [gnugo-devel] FW: Owl tuning help needed

[Gunnar Farneback]

Arend wrote:
> This huge delta convinces me that we need to automatically watch for
> performance regressions in the same way as we are already watching for
> move choice regressions. (Just counting difference in owl nodes for
> owl.tst just doesn't suffice, e.g. Also your numbers show so much
> variation that any single test suite is not enough.)

Yes, this would be good to do.

> The way I'd suggest to do this: Create a new symbol for regress.awk,
> e.g. as follows:
> 
> # Report number of nodes visited by the tactical reading
> 10000 get_reading_node_counter
> #? [54321]!
> 
> regress.awk would then report:
> 10000: old value '54321', new value '65432'.
> 
> (Or maybe s.th. fancier like computing the relative difference in %,
> if someone knows awk better than me. And maybe even compute the total
> number of trymove/reading/owl nodes.).

But I don't think we should try to add it to the basic awk script or
start updating node numbers in the .tst files.

A better solution would be to have a more high-level regress script
which could do more detailed node counting and generate fancy reports
of both results and node counts.

What I really would like is a system that automatically runs
regressions (including node counts) for every CVS checkin (or a batch
of checkins if they come close together) and presents results on the
web. This would not need to be as detailed and informative as Trevor's
pages, but with history so it's easy to follow results over time.

Possibly the Xenofarm project at
http://pike.ida.liu.se/applications/xenofarm/ could be useful.

/Gunnar