[Gnushogi-devel] Comparing GNUShogi 1.2 and later versions

[ydirson]

I resumed a bit of work on my 1.2-revival branch (now pushed to Savannah), in 
order
to make it buildable with "gcc-6 -m32" (64bit was probably not the target at 
that time,
and in fact the tests segfault on x86_64, although there are build options for 
alpha...).

I launched a pair of games between that 1.2p03 (even without opening moves) and 
1.5 snapshot,
and the results are as interesting as I imagined they could be:

* 1.2p03 wins all the games
* 1.5 even lets its clock expire quite often
* xshogi UI blocks when it gets "White mates" from engine so it takes manual 
work to extract
  the games

Games recorded with:
 xshogi -debug True -fsp gnushogi -ssp ./gnushogix-1.2pl03
Output then processed with to get what looks like a decent PGN:
 grep "Received from first:" |grep -v time|cut -d: -f2|tail -n+3

Sample games attached.  Sente is 1.5 (despite filename, sorrry), Gote is 1.2p03

Game 1: sente gets invaded at move 10 already, and could give up around move 20
Game 2: sente gets seriously invaded only at move 25, and what follows does 
look like crap
 (did not take the time to analyse early moves anyway, my own level is so weak 
:)
Game 3: sente invaded at move 15

Looks like we have a pattern here :)


In further tests:

1.3.2 vs 1.2p03 (no book on either side):
 1. stength difference is not as drastic, but 1.3.2 still looses on time (game 
goes on, it would look like 1.3.2 gets slowly overwhelmed, see game)
 2. 1.3.2 still looses on time, "but" gets ahead and mates - strange things 
happen, though, like the 37-38 moves

1.5pre vs 1.3.2 (no book for 1.3.2):
 1. 1.3.2 wins after 75 moves (no clock expiration on either side)
 2. 1.5 wins after 88 moves (xshogi says "black's flag has fallen", but I'm not 
sure what it's supposed to be, no clock is negative here)


My conclusions so far:
* there may still be some remaining hope in this engine
* real benchmarking has to be done to identify when the various regressions 
occurred (controlled conditions,
  32 vs 64bits, book or not)
* then we'll know where it stands more precisely

13vs12.2.log
Description: Text Data

15vs13.2.pgn
Description: application/chess-pgn

15vs13.1.pgn
Description: application/chess-pgn

13vs12.1.pgn
Description: application/chess-pgn

14vs12.1.pgn
Description: application/chess-pgn

14vs12.2.pgn
Description: application/chess-pgn

14vs12.3.pgn
Description: application/chess-pgn