GNU Chess bug report: Time trouble with external book

[Kirill Kryukov]

Dear GNU Chess developers,

I would like to report a bug in GNU Chess 5.07. The bug results in
serious time trouble when GNU Chess is playing with esternal book
controlled by a GUI, when playing with 40/40 type time control (also
with any other X/Y time control). GNU chess does not consider the
moves operated by a GUI as part of the time control session. As a
result its move counter is lagging behind the real move count. This
results in a huge time trouble after about move 50. The exact
mexhanism of this bug is explained by H.G. Muller (copied from
Winboard Forum:
http://www.open-aurec.com/wbforum/viewtopic.php?p=188279#p188279):

"
Many WB engines suffer from this when you play from an external opening book.
They then receive the go command later in the game, (say at move 5),
and start counting
the moves only from there. So when they reach move 40, to them it is
only move 35.
The extra time on their clock that the GUI reports to them then comes
as a complete surprise
to them, which they happily exploit during the remaining 5 moves, so
that they have used up
nearly all time at move 45 (which they think is move 40). It then
comes again as a complete
surprise (this time an unpleasant one) that they do not receive the
expected new time
quota after this move, so that in fact they have to play the remaining
35 moves of the second
session in the few seconds that were left on their clock after move 45
(which they thought
to be 40). On move 75/80 the whole thing repeats. They thus play the
first 5 moves of any
session except the first in the full time for the session, and the
other 35 moves in just a few
seconds. Of course this means they usually do not survive upto the
third session...

In principle this behavior also occurs under WinBoard. But the more
book moves you have,
the less obvious the effect is. This because most engines manage their
time such that they
normally do leave a safety margin of about 7 moves before any
time-control point. This
because a move in a difficult position can easily take 5 times as long
as an average move.
So if you had 24 moves in book, the engine realizes its error only on
move 64, (at 40/40),
and then have to do the remaining 16 moves before it gets new time in
7 min, which is not
extremely much faster than normal, and usually interpreted as that the
engine speeds up
in the end-game under time pressure.

That this is so common is due to a very unclear sn confusing
description of the clock issue in
the WB protocol document on Tim Mann's website. I tried to clarify it
in the protocol definitin
here, based on how WB did actually manage the clocks. There is no easy
work-around for engines
that suffer from this, other than play them with external book at
incremental time controls
only.
"

In fact a number of Winboard engines are suffering from this same bug.
Some discussions about this:
http://www.open-aurec.com/wbforum/viewtopic.php?t=49792
http://www.open-aurec.com/wbforum/viewtopic.php?t=50346

Now I just want to stress the seriousness of the bug. Basically it
hugely handicaps an engine in tournaments with external books.

Large number of computer chess enthusiasts like to compare engines
without own books. There are multiple reasons for this:
1. Some engines include huge opening books created by professionals.
These books basically ensure engine's advantage from the opening. For
many it is interesting to compare the chess algorithms rather than
human-assembled opening knowledge.
2. Many people use engines to analyze their games, to study the
openings, study the games of others, etc.. This typically involves
using a large game database, and an engine is only used as analysis
tool. For such uses engine's own book is totally irrelevant, but
engine's analysis quality (which correlates well with the playing
strength) is important. So for these users it is important to know how
strong is the engine on its own, without own book.
3. As the opening theory develops, new good lines are found, some old
moves are refuted, etc.. So this year book is likely to be stronger
than the last year one. Some people feel it's not fair to punish the
older engines just because their book is outdated.

There may be more reasons, in any case there is significant interest
in engine performances without own books. However if two engines will
play a match without any opening books, they will repeat the same game
over and over. (Actually there may be some variations, but anyway a
match of 100 games will have many repeated games). The solution is to
use a short external book, same for both engines in a match. The
opening is taken randomly from the book, then the engines play it out.
Often a limit is placed on book length. I use books up to 8 moves deep
in my tournament. CCRL limits the book to 12 moves.

Some important rating lists and tournaments are played this way (with
a short general book shared by all engines):

CCRL 40/40: http://computerchess.org.uk/ccrl/4040/
CCRl 40/4: http://computerchess.org.uk/ccrl/404/
CEGT 40/20: http://www.husvankempen.de/nunn/rating.htm
CEGT 40/4: http://www.husvankempen.de/nunn/blitz.htm
KCEC: http://kirill-kryukov.com/chess/kcec/

Now, what happens when an engine with this bug plays in such
tournament. There are two possibilities for such engine to play:

1. Play with external book like all other engines. In this case the
engine is handicapped by having to move instantly after about move 50.
How much is the performance decrease? It really depends on an engine.
Some, like Resp, can't play this way at all - they lose on time every
game. Some, like EXchess, seem to be almost OK, because they keep some
time reserve. How much GNU Chess is punished by this bug remains to be
seen. I may run it both ways and compare the results, but it will take
some time (It needs many games).

2. Play without any opening book. In this case the engine is still
handicapped since the opponent is using an external book. Book moves
are usually better than what an engine can come up with on its own (at
least more often better than not). Also book saves some thinking time.
My estimation is that a short 8-move opening book (general book, not
tuned to particular engine) is worth about 60 Elo points. It's a rough
estimate, I did not study this systematically. It also depends on an
engine. With a strong engine there is less benefit from a book, with
weaker engines the book benefit is probably larger.

In both of these two cases the real strength of an engine can not be
measured. The ideas, algorithms and knowledge put into the engine can
not be evaluated properly because of this subtle protocol issue.

I hope it is easy to fix. If I can help with anything, or provide any
additional information, please let me know. I don't subscribe to the
list, so please use email to contact me.

Best wishes,
Kirill