Re: [gnugo-devel] GNU Go 3.3.23

[Paul Pogonyshev]

Dan wrote:
> Marco wrote:
> > I haven't installed pike yet, but if anyone cares and since the runs
> > completed, here are the results when running 3.3.23 and 3.2 on 3.2
> > all_batches:
> >
> > 3.3.23  7110.800u 40.070s 4:14:11.24 46.8%      0+0k 0+155io 0pf+0w
> > 3.2     7763.120u 38.030s 4:11:50.20 51.6%      0+0k 0+161io 0pf+0w
>
> This seems to show 3.3.23 ripping through a much longer regression
> faster than 3.2 goes through a shorter one.

i think Marco meant he ran both versions on 3.2 regressions (the same
tests).

> Yet I just finished a 75 game series with twogtp.pike which
> shows 3.3.23 at level 10 as taking MUCH more time than 3.2.
>
> I ran this at night with the computer not used otherwise except to
> serve web pages.  These are 3 stone games. Black is GNU Go 3.2,
> W is 3.3.23.  I think we can safely say that 3.3.23 is not 3 stones
> stronger than 3.2.
>
> Total 75 game(s).
> White won 28. Black won 47.
> Engines disagreed on results of 2 game(s).
> White: 33673.330s CPU time.
>
> Time control report (wall move generation time):
>   White: 557:01.3
>   Black: 431:24.7
>
> One thing is that in the regressions, the persistent caches
> are less used than in real games. I can't quite imagine that
> that accounts for this.

i'd guess that it's mostly break-in code, but i'm not sure.  have you
tried level 9 against 3.2?  and btw, let's not forget that we also have
a much slower connection code and somewhat slower semeai in this version.
however, those should be (somewhat) compensated with all the speedups
we had.

btw, if you run twogtp.pike with some time control options (e.g. -t 60
which shouldn't affect the engines in any way), and in verbose mode,
you can see that in beginning, 3.3.23 plays faster than 3.2.  i'm not
exactly sure what that means - either as you suggested that persistent
cache is somehow less efficient, or that break-in code takes a lot of
time when some territories appear on the board (i.e. not in the
beginning).

> Is GNU Go 3.3.23 too slow to release?

i don't think it should stop us from releasing 3.4.  of course it's
better to be faster, but even if there is a 25% slowdown, we should go
ahead.  hardware has certainly improved over this year.

Paul