[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-gnubg] Bug in sigmoid?

From: Olivier Baur
Subject: Re: [Bug-gnubg] Bug in sigmoid?
Date: Sat, 19 Apr 2003 16:23:25 +0200

Le vendredi, 18 avr 2003, à 23:01 Europe/Paris, Joseph Heled a écrit :

I am not sure about that. I have approximated exp(x) with 1+x, which is mathematically sound and has an o(x^2) error, which is nice in the [01] range the approximation is used. I think this is not the case for 1/(1+x)

Yes, the approximation used for exp(x) in the current sigmoid function is correct and quite clever, since we have exp(x) = exp(X + dx) = exp(X).exp(dx) = exp(X).(1 + dx) + o(dx^2), which requires only one exp(X) lookup.

Then, I don't understand what you mean about 1/(1+x)...

Second, we need to re-train (we can't keep 2 nets around for practical reasons).

Can't agree less :-)
And, apart from not being myself able to train a new net :-) , that's the reason why I used the current sigmoid function approximation to build the sigmoid2 lookup table, so we get the same results.

Third, I need to see numbers on how faster this is on non scalar, regular x86 machine.

I don't know if this is a typo, but please note a "scalar" processor is what you call a "regular" processor; on the other hand, "non-scalar" and "super-scalar" refer to vector-computing.

Please note the speed increase I have measured in sigmoid2 (+60%) was for a regular scalar (ie *non* vector) implementation (on a PPC G4 processor); with a vector implementation of sigmoid2 (on the same processor), I actually got a whopping +250% speed increase...

So let me know what figures you get on a scalar x86 :-)

-- Olivier

reply via email to

[Prev in Thread] Current Thread [Next in Thread]