[gnugo-devel] arend_1

gnugo-devel
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnugo-devel] arend_1_26.3 tested

From:	Daniel Bump
Subject:	[gnugo-devel] arend_1_26.3 tested
Date:	Thu, 21 Feb 2002 07:32:42 -0800
I tested the patch arend_1_26.3 by running the regressions
both with and without the patch. I ran all regressions with
both patched and unpatched versions simultaneously on a
dual processor machine with two 433Mh Celerons and 128 Mb
RAM. Then I reran just the owl test with only one process.

One would expect a greater speedup with owl.tst than
with all_batches but I found the reverse. The speedup
for all_batches was 4.45 % and the speedup for owl.tst
was 2.29 %. There could be some randomness in the timing
results but I'm convinced this patch is a good speedup.

On the down side there was a net loss of eight tests.
Unpatched, we are ahead of the regressions by 5 tests.
(This is due to the fact that these tests were run with
experimental influence because none of the patches since
3.1.25 should have much effect on the engine without
experimental influence.) After the patch, we are behind
by 3 tests.

My conclusion is that we want this patch but we should look
a little harder at the failures.

Patches in the CVS since 3.1.25:

arend_1_26.4. Verbosity. 
scoring_1_26.1. Bugfix in close_bubbles. 
autolevel_1_26.1. Bugfix in autolevel 
gunnar_1_26.2. Semeai bugfix. 
gunnar_1_26.1. Revised "stepped on a bug" message. 
arend_1_26.2. Use old influence for deciding which owls to examine.
trevor_1_26.1. Alternative non-variadic trace. 
arend_1_26.1. Experimental influence speedup. 
trevor_1_25.4. Stack assertion. Arrived too late to go in 3.1.25. 

Results from the CVS version + arend_1_26.3 with configure options:
experimental influence
experimental connections
alternate connections

40160 seconds: make all_batches
469   seconds: make owl

56 PASS
59 FAIL

./regress.sh . reading.tst
./regress.sh . owl.tst
23 unexpected FAIL: Correct '0', got '1 S11'
132 unexpected PASS!
./regress.sh . owl_rot.tst
66 unexpected PASS!
./regress.sh . ld_owl.tst
./regress.sh . optics.tst
./regress.sh . filllib.tst
./regress.sh . atari_atari.tst
./regress.sh . connection.tst
./regress.sh . blunder.tst
./regress.sh . trevora.tst
450 unexpected PASS!
470 unexpected FAIL: Correct '!E2', got 'E2'
530 unexpected PASS!
./regress.sh . strategy.tst
20 unexpected PASS!
22 unexpected FAIL: Correct 'K3', got 'J11'
27 unexpected FAIL: Correct 'C3', got 'F2'
34 unexpected FAIL: Correct 'E17', got 'N15'
45 unexpected FAIL: Correct 'G17|G18|C1', got 'H1'
./regress.sh . endgame.tst
./regress.sh . heikki.tst
./regress.sh . neurogo.tst
20 unexpected FAIL: Correct '1 S16', got '1 T18'
./regress.sh . arb.tst
./regress.sh . rosebud.tst
1 unexpected FAIL: Correct 'E16', got 'E18'
./regress.sh . golife.tst
4 unexpected PASS!
./regress.sh . arion.tst
./regress.sh . viking.tst
3 unexpected FAIL: Correct 'M15|B7', got 'O4'
7 unexpected PASS!
9 unexpected PASS!
./regress.sh . ego.tst
./regress.sh . dniwog.tst
./regress.sh . lazarus.tst
4 unexpected FAIL: Correct 'R12|Q12|M8', got 'T5'
7 unexpected PASS!
14 unexpected FAIL: Correct 'Q15|T5|H5', got 'R15'
16 unexpected PASS!
./regress.sh . trevorb.tst
180 unexpected FAIL: Correct 'L12', got 'H3'
250 unexpected FAIL: Correct 'C2', got 'L8'
440 unexpected FAIL: Correct 'E6', got 'J6'
450 unexpected PASS!
500 unexpected FAIL: Correct '!J11', got 'J11'
600 unexpected PASS!
660 unexpected FAIL: Correct 'D3', got 'H5'
700 unexpected PASS!
790 unexpected FAIL: Correct 'K6|K5', got 'A6'
840 unexpected PASS!
./regress.sh . strategy2.tst
51 unexpected FAIL: Correct 'P3', got 'N5'
82 unexpected FAIL: Correct 'L5', got 'Q17'
85 unexpected FAIL: Correct 'O2', got 'N5'
90 unexpected FAIL: Correct 'B14|D17', got 'H2'
91 unexpected FAIL: Correct 'B14|G13', got 'H2'
./regress.sh . nicklas1.tst
./regress.sh . nicklas2.tst
./regress.sh . nicklas3.tst
./regress.sh . nicklas4.tst
./regress.sh . nicklas5.tst
1202 unexpected PASS!
./regress.sh . manyfaces.tst
1 unexpected FAIL: Correct 'S13', got 'N15'
./regress.sh . niki.tst
4 unexpected FAIL: Correct 'Q8', got 'R13'
./regress.sh . trevor.tst
190 unexpected FAIL: Correct 'B7', got 'F5'
200 unexpected PASS!
./regress.sh . tactics.tst
./regress.sh . buzco.tst
8 unexpected PASS!
./regress.sh . nngs.tst
230 unexpected PASS!
310 unexpected PASS!
420 unexpected FAIL: Correct '!C4|D3|C3|B3', got 'C4'
630 unexpected FAIL: Correct '!J16', got 'J16'
640 unexpected PASS!
680 unexpected PASS!
700 unexpected FAIL: Correct 'K17', got 'M2'
710 unexpected FAIL: Correct '!B12', got 'B12'
800 unexpected FAIL: Correct '!N19|M19|L19|S18|K1', got 'K1'
850 unexpected PASS!
900 unexpected FAIL: Correct 'Q15', got 'Q7'
1020 unexpected PASS!
1050 unexpected PASS!
1080 unexpected FAIL: Correct 'K17', got 'F3'
1250 unexpected PASS!
1520 unexpected FAIL: Correct 'F15|F14', got 'F12'
1660 unexpected PASS!
1690 unexpected FAIL: Correct 'K17|L16', got 'D8'
1760 unexpected PASS!
1790 unexpected PASS!
1800 unexpected PASS!
1940 unexpected PASS!
1955 unexpected FAIL: Correct 'D3', got 'R4'
./regress.sh . trevorc.tst
170 unexpected PASS!
330 unexpected PASS!
340 unexpected FAIL: Correct 'M3|M4', got 'L2'
360 unexpected PASS!
410 unexpected PASS!
530 unexpected FAIL: Correct 'J3', got 'K4'
600 unexpected PASS!
610 unexpected PASS!
750 unexpected PASS!
1040 unexpected FAIL: Correct '!A8', got 'A8'
1240 unexpected PASS!
1250 unexpected PASS!
1290 unexpected PASS!
1310 unexpected PASS!
1340 unexpected PASS!
1350 unexpected PASS!
1370 unexpected FAIL: Correct 'B9', got 'C9'
1400 unexpected PASS!
1470 unexpected FAIL: Correct 'L10', got 'L12'
1570 unexpected FAIL: Correct 'E2|F2|F3', got 'E1'
1610 unexpected PASS!
1620 unexpected FAIL: Correct '!E11', got 'E11'
1640 unexpected PASS!
./regress.sh . strategy3.tst
108 unexpected PASS!
./regress.sh . capture.tst
./regress.sh . connect.tst
./regress.sh . global.tst
1 unexpected FAIL: Correct 'B3', got 'F5'
5 unexpected FAIL: Correct 'O4', got 'S12'
8 unexpected FAIL: Correct 'H9', got 'G11'
28 unexpected PASS!
./regress.sh . vie.tst
./regress.sh . arend.tst
14 unexpected FAIL: Correct 'P17', got 'G13'
24 unexpected FAIL: Correct '!P16', got 'P16'
32 unexpected FAIL: Correct 'F9|E8', got 'G8'
33 unexpected FAIL: Correct 'F9', got 'M5'
./regress.sh . 13x13.tst
17 unexpected FAIL: Correct 'L5|L3', got 'G8'
33 unexpected PASS!
38 unexpected FAIL: Correct 'G11', got 'G10'
39 unexpected FAIL: Correct 'H4|J4', got 'E2'
41 unexpected FAIL: Correct '!C8', got 'C8'
43 unexpected FAIL: Correct 'D6', got 'B6'
76 unexpected FAIL: Correct 'K6|L6|J3', got 'J5'
85 unexpected FAIL: Correct 'D2', got 'B10'
88 unexpected PASS!
./regress.sh . semeai.tst
./regress.sh . trevord.tst
120 unexpected FAIL: Correct 'B9', got 'F7'
230 unexpected PASS!
240 unexpected PASS!
550 unexpected FAIL: Correct 'T8', got 'J8'
830 unexpected PASS!
./regress.sh . strategy4.tst
200 unexpected PASS!
204 unexpected FAIL: Correct 'B2', got 'G6'
207 unexpected PASS!
209 unexpected PASS!

Results from CVS version without arend_1_26.3

configure option enabled: experimental influence
configure option enabled: experimental connections
configure option enabled: alternate connections

42033 seconds: make all_batches
480   seconds: make owl

49 PASS
44 FAIL

./regress.sh . reading.tst
./regress.sh . owl.tst
./regress.sh . owl_rot.tst
./regress.sh . ld_owl.tst
./regress.sh . optics.tst
./regress.sh . filllib.tst
./regress.sh . atari_atari.tst
./regress.sh . connection.tst
./regress.sh . blunder.tst
./regress.sh . trevora.tst
450 unexpected PASS!
470 unexpected FAIL: Correct '!E2', got 'E2'
530 unexpected PASS!
./regress.sh . strategy.tst
18 unexpected FAIL: Correct 'E4|F3', got 'N11'
20 unexpected PASS!
22 unexpected FAIL: Correct 'K3', got 'J11'
./regress.sh . endgame.tst
./regress.sh . heikki.tst
./regress.sh . neurogo.tst
./regress.sh . arb.tst
./regress.sh . rosebud.tst
1 unexpected FAIL: Correct 'E16', got 'E18'
./regress.sh . golife.tst
4 unexpected PASS!
./regress.sh . arion.tst
./regress.sh . viking.tst
3 unexpected FAIL: Correct 'M15|B7', got 'O4'
7 unexpected PASS!
9 unexpected PASS!
./regress.sh . ego.tst
./regress.sh . dniwog.tst
./regress.sh . lazarus.tst
4 unexpected FAIL: Correct 'R12|Q12|M8', got 'T5'
7 unexpected PASS!
16 unexpected PASS!
./regress.sh . trevorb.tst
180 unexpected FAIL: Correct 'L12', got 'H3'
250 unexpected FAIL: Correct 'C2', got 'L8'
440 unexpected FAIL: Correct 'E6', got 'J6'
450 unexpected PASS!
500 unexpected FAIL: Correct '!J11', got 'J11'
600 unexpected PASS!
660 unexpected FAIL: Correct 'D3', got 'H5'
700 unexpected PASS!
790 unexpected FAIL: Correct 'K6|K5', got 'A6'
840 unexpected PASS!
./regress.sh . strategy2.tst
51 unexpected FAIL: Correct 'P3', got 'N5'
85 unexpected FAIL: Correct 'O2', got 'N5'
90 unexpected FAIL: Correct 'B14|D17', got 'H2'
91 unexpected FAIL: Correct 'B14|G13', got 'H2'
./regress.sh . nicklas1.tst
./regress.sh . nicklas2.tst
./regress.sh . nicklas3.tst
./regress.sh . nicklas4.tst
./regress.sh . nicklas5.tst
1202 unexpected PASS!
./regress.sh . manyfaces.tst
1 unexpected FAIL: Correct 'S13', got 'N15'
./regress.sh . niki.tst
4 unexpected FAIL: Correct 'Q8', got 'R13'
./regress.sh . trevor.tst
190 unexpected FAIL: Correct 'B7', got 'F5'
./regress.sh . tactics.tst
./regress.sh . buzco.tst
8 unexpected PASS!
./regress.sh . nngs.tst
230 unexpected PASS!
310 unexpected PASS!
420 unexpected FAIL: Correct '!C4|D3|C3|B3', got 'C4'
640 unexpected PASS!
680 unexpected PASS!
700 unexpected FAIL: Correct 'K17', got 'K18'
800 unexpected FAIL: Correct '!N19|M19|L19|S18|K1', got 'K1'
850 unexpected PASS!
900 unexpected FAIL: Correct 'Q15', got 'Q7'
1020 unexpected PASS!
1050 unexpected PASS!
1080 unexpected FAIL: Correct 'K17', got 'F3'
1250 unexpected PASS!
1660 unexpected PASS!
1690 unexpected FAIL: Correct 'K17|L16', got 'D8'
1760 unexpected PASS!
1790 unexpected PASS!
1800 unexpected PASS!
1940 unexpected PASS!
1955 unexpected FAIL: Correct 'D3', got 'R4'
./regress.sh . trevorc.tst
170 unexpected PASS!
330 unexpected PASS!
360 unexpected PASS!
410 unexpected PASS!
530 unexpected FAIL: Correct 'J3', got 'K4'
600 unexpected PASS!
610 unexpected PASS!
750 unexpected PASS!
1040 unexpected FAIL: Correct '!A8', got 'A8'
1080 unexpected FAIL: Correct '!L8|K10|K9', got 'L8'
1240 unexpected PASS!
1290 unexpected PASS!
1310 unexpected PASS!
1340 unexpected PASS!
1350 unexpected PASS!
1370 unexpected FAIL: Correct 'B9', got 'C9'
1400 unexpected PASS!
1570 unexpected FAIL: Correct 'E2|F2|F3', got 'E1'
1640 unexpected PASS!
./regress.sh . strategy3.tst
./regress.sh . capture.tst
./regress.sh . connect.tst
./regress.sh . global.tst
5 unexpected FAIL: Correct 'O4', got 'S12'
28 unexpected PASS!
./regress.sh . vie.tst
./regress.sh . arend.tst
14 unexpected FAIL: Correct 'P17', got 'G13'
24 unexpected FAIL: Correct '!P16', got 'P16'
32 unexpected FAIL: Correct 'F9|E8', got 'G8'
33 unexpected FAIL: Correct 'F9', got 'M5'
./regress.sh . 13x13.tst
17 unexpected FAIL: Correct 'L5|L3', got 'G8'
33 unexpected PASS!
38 unexpected FAIL: Correct 'G11', got 'G10'
41 unexpected FAIL: Correct '!C8', got 'C8'
43 unexpected FAIL: Correct 'D6', got 'B6'
76 unexpected FAIL: Correct 'K6|L6|J3', got 'J5'
85 unexpected FAIL: Correct 'D2', got 'B10'
88 unexpected PASS!
./regress.sh . semeai.tst
./regress.sh . trevord.tst
240 unexpected PASS!
550 unexpected FAIL: Correct 'T8', got 'J8'
830 unexpected PASS!
./regress.sh . strategy4.tst
190 unexpected PASS!
204 unexpected FAIL: Correct 'B2', got 'G6'
207 unexpected PASS!
209 unexpected PASS!

Dan
[Prev in Thread]
Current Thread
[Next in Thread]
[gnugo-devel] arend_1_26.3 tested, Daniel Bump <=
- Re: [gnugo-devel] arend_1_26.3 tested, Arend Bayer, 2002/02/22
Prev by Date: Re: [gnugo-devel] 100 game series
Next by Date: Re: [gnugo-devel] arend_1_26.3: owl speedup
Previous by thread: [gnugo-devel] arend_1_26.4: fix of arend_1_25.2
Next by thread: Re: [gnugo-devel] arend_1_26.3 tested
Index(es):
- Date
- Thread