Re: [Gnushogi-devel] Comparing GNUShogi 1.2 and later versions

[h . g . muller]

Well, I did a quick test with GNU Shogi 1.4.1. I had it play 30 games at
40 move/min against CrazyWa 1.0.5. CrazyWa won that by 27-3. Then I played
10 games at 40 moves/5 min; sometimes engines with more knowledge do
better at slower TC, where they can avoid the coursest tactical mistakes.
CrazyWa won that by 9-1, the points for GNU Shogi being awarded by two
draw adjudications after it crashed (presumably because some maximum game
length was exceeded) in impasse situations.

I am not sure how fair the test was; I set CrazyWa for using 64MB hash
table, but through "ps l" I noticed that GNU Shogi was using less (14MB vs
70MB memory footprint):

address@hidden:~/uci2wb$ ps l
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
0  1000  1458  1430  20   0  21752  4556 wait   Ss   pts/0      0:01 bash
0  1000  1821  1430  20   0  21752  4548 wait   Ss   pts/1      0:01 bash
0  1000  5549  1458  20   0 187556 26484 poll_s S+   pts/0      2:03
xboard -var
0  1000  5866  5549  20   0  14552   688 pipe_w Sl+  pts/0      0:00
../uci2wb/x
0  1000  5867  5866  20   0  14064  8132 -      R+   pts/0      5:33
./gnushogi
0  1000  5869  5549  20   0  70456 66956 pipe_w S+   pts/0      1:50
./crazywa
0  1000  5873  1821  20   0   6840   992 -      R+   pts/1      0:00 ps l

I don't know how to control the hash usage in GNU Shogi; I never really
used it before. Playing strength is usually not very dependent on hash
size, however. (In Chess engines speed typically goes as the 12th root of
hash size.)

What is worse, is that GNU Shogi seems to cheat in time usage: the above
'ps l' was taken after both engines had used ~4 min each on their XBoard
clocks. It seems that of those 4 min, CrazyWa got less than 2, while GNU
Shogi got 5.5! (XBoard also shows a lot of CPU usage, but that was for all
40 games together, while the engine processes are restarted for every
game.) So it seems GNU Shogi has been pondering, competing with CrazyWa
for CPU while the latter was thinking. I don't know how this is possible,
as I was conducting the games with ponder off, and I verified that XBoard
has been sending the 'easy' command through XS2WB to GNU Shogi. In
addition, my laptop has two cores (4 hyper-threads), and wasn't running
any other heavy calculation, so in principle running two programs at the
same time should not need them to be scheduled only a fraction of the
time. Perhaps the VM on which I run Linux only allows usage of a single
core.

Anyway, CazyWa wins this convincingly. In particular, it is much more
aggressive towards the King, and often sacrifices lots of material to draw
it into the open. (Which occasionally backfires.) I attached a PGN file
with the games. The score comments in there are CrazyWa's evaluations; I
have not figured out how I can make GNU Shogi 1.4.1 print scores and PVs.
The 'post' command did not seem to have any effect. (At least, it did not
switch on CECP-type thinking output.)

I also attached a somewhat cleaned-up (deleting dead code) version of
XS2WB.c, which now also communicates the name of the engine (assumed to be
the first line it prints) to the GUI (so that the GNU-Shogi version number
will be automatically included in the PGN).

Regards, H.G.

gnushogi.pgn
Description: application/chess-pgn

XS2WB.c
Description: Text Data