gnushogi-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gnushogi-devel] Comparing GNUShogi 1.2 and later versions


From: H.G. Muller
Subject: Re: [Gnushogi-devel] Comparing GNUShogi 1.2 and later versions
Date: Sun, 18 Feb 2018 15:30:08 +0100
User-agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0

It would probably be more revealing to try GNU Shogi 1.2 against an engine known to be reasonably bug-free. This might be difficult, however, as all available engines use XBoard protocol, and I suppose GNU Shogi 1.2 is not yet patched for that, and is still stuck with xShogi protocol, for which there are no opponents at all. Otherwise it would be a good idea to try it against CrazyWa (which is virtually knowledgeless; it has for instance no idea what generals are, and that it is good strategy to keep some of those close to your King). Or even one of the configurable chess-variant engines, such as Sjaak II, configured for Shogi (which also have zero specific Shogi knowledge). If GNU Shogi 1.2. is totally crushed by those (which would not surprise
me), finding out exactly where it regressed seems a waste of time.

@Yann:
BTW, I did finally manage to get a decently looking source code for CrazyWa that works at least as well as the versions that participated in the Crazyhouse championships Dec. 2016 and 2017. I tested it against that old versions in Crazyhouse and Shogi, and it scores about 50%, as expected. (In Crazyhouse even 60%, as I fixed an overflow bug there that wouldn't affect Shogi.) I did not explicitly test it again in mini-Shogi, Judkin's Shogi, Tori Shogi or Wa Shogi, but there is no reason to suspect these could be broken without breaking all variants. So if you are interested and have time, you could package the latest snapshot (version 1.0.5) from the repository at http://hgm.nubati.net . I added a Makefile with 'make install'
and 'make dist' targets. I did not add any Wa- or Tori-Shogi piece graphics.

Regards,
H.G

Op 2/11/2018 om 10:29 PM schreef address@hidden:
I resumed a bit of work on my 1.2-revival branch (now pushed to Savannah), in 
order
to make it buildable with "gcc-6 -m32" (64bit was probably not the target at 
that time,
and in fact the tests segfault on x86_64, although there are build options for 
alpha...).

I launched a pair of games between that 1.2p03 (even without opening moves) and 
1.5 snapshot,
and the results are as interesting as I imagined they could be:

* 1.2p03 wins all the games
* 1.5 even lets its clock expire quite often
* xshogi UI blocks when it gets "White mates" from engine so it takes manual 
work to extract
   the games

Games recorded with:
  xshogi -debug True -fsp gnushogi -ssp ./gnushogix-1.2pl03
Output then processed with to get what looks like a decent PGN:
  grep "Received from first:" |grep -v time|cut -d: -f2|tail -n+3

Sample games attached.  Sente is 1.5 (despite filename, sorrry), Gote is 1.2p03

Game 1: sente gets invaded at move 10 already, and could give up around move 20
Game 2: sente gets seriously invaded only at move 25, and what follows does 
look like crap
  (did not take the time to analyse early moves anyway, my own level is so weak 
:)
Game 3: sente invaded at move 15

Looks like we have a pattern here :)


In further tests:

1.3.2 vs 1.2p03 (no book on either side):
  1. stength difference is not as drastic, but 1.3.2 still looses on time (game 
goes on, it would look like 1.3.2 gets slowly overwhelmed, see game)
  2. 1.3.2 still looses on time, "but" gets ahead and mates - strange things 
happen, though, like the 37-38 moves

1.5pre vs 1.3.2 (no book for 1.3.2):
  1. 1.3.2 wins after 75 moves (no clock expiration on either side)
  2. 1.5 wins after 88 moves (xshogi says "black's flag has fallen", but I'm 
not sure what it's supposed to be, no clock is negative here)


My conclusions so far:
* there may still be some remaining hope in this engine
* real benchmarking has to be done to identify when the various regressions 
occurred (controlled conditions,
   32 vs 64bits, book or not)
* then we'll know where it stands more precisely





---
This email has been checked for viruses by AVG.
http://www.avg.com




reply via email to

[Prev in Thread] Current Thread [Next in Thread]