[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-gnubg] Possible error in SSE sigmoid calculation
From: |
Philippe Michel |
Subject: |
Re: [Bug-gnubg] Possible error in SSE sigmoid calculation |
Date: |
Wed, 21 Dec 2011 14:03:57 +0100 (CET) |
User-agent: |
Alpine 2.00 (BSF 1167 2008-08-23) |
On Wed, 21 Dec 2011, Joseph Heled wrote:
I find it hard to believe.
There is a simple test - if the two implementations give the same vale for
the same position, ply 0.
Analyzing and exporting to text a sample match at 0-ply, I get this as a
rough measure of number of discrepancies between the following options :
nosse : scalar code without SSE intrinsics
ssecvt : SSE intrinsics with cvtps instruction
ssecvtt : SSE intrinsics with cvttps intruction
% diff nosse.txt ssecvt.txt | wc
16360 94346 681212
% diff nosse.txt ssecvtt.txt | wc
2434 12056 90952
The differences between nosse and ssecvtt are due, I think, to the usage
of a reciprocal instead of a true division in 1/(1+exp(x)). I suppose this
was found acceptable at the time and would dissappear if everything was
compiled with the -ffast-math flag. Maybe one could use something like
#ifdef __FAST_MATH__
... use reciprocal intrinsic
#else
... use divide intrinsic
#endif
in neuralnetsse.c
The discrepancies in ssecvt, on the other hand, are not only about seven
times more common but come from the flawed approximation of exp(x) in
addition to the above shortcut.
Looking cursorily at the exports, I found only occasionnal 0.001
differencies in the ssecvtt case, without moves swapped.
In the ssecvt case, there were some almost-equal moves ordered differently
(in the exported top five choices). I didn't found cases where the top
move was changed or equity differences of more than 0.001, but I'm
convinced the former, at least, could happen.
These are still pretty small differences. In the final summary of these
analyses for instance I get :
nosse : Error total EMG (MWC) -4.142 (-14.676%) -4.845 (-15.479%)
ssecvt : Error total EMG (MWC) -4.142 (-14.682%) -4.828 (-15.344%)
ssecvtt : Error total EMG (MWC) -4.143 (-14.683%) -4.846 (-15.484%)