[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## Re: [Bug-gnubg] Bug in sigmoid?

**From**: |
Olivier Baur |

**Subject**: |
Re: [Bug-gnubg] Bug in sigmoid? |

**Date**: |
Sat, 19 Apr 2003 16:23:25 +0200 |

Le vendredi, 18 avr 2003, à 23:01 Europe/Paris, Joseph Heled a écrit :

`I am not sure about that. I have approximated exp(x) with 1+x, which
``is mathematically sound and has an o(x^2) error, which is nice in the
``[01] range the approximation is used. I think this is not the case for
``1/(1+x)
`

`Yes, the approximation used for exp(x) in the current sigmoid function
``is correct and quite clever, since we have exp(x) = exp(X + dx) =
``exp(X).exp(dx) = exp(X).(1 + dx) + o(dx^2), which requires only one
``exp(X) lookup.
`
Then, I don't understand what you mean about 1/(1+x)...

`Second, we need to re-train (we can't keep 2 nets around for practical
``reasons).
`

Can't agree less :-)

`And, apart from not being myself able to train a new net :-) , that's
``the reason why I used the current sigmoid function approximation to
``build the sigmoid2 lookup table, so we get the same results.
`

`Third, I need to see numbers on how faster this is on non scalar,
``regular x86 machine.
`

`I don't know if this is a typo, but please note a "scalar" processor is
``what you call a "regular" processor; on the other hand, "non-scalar"
``and "super-scalar" refer to vector-computing.
`

`Please note the speed increase I have measured in sigmoid2 (+60%) was
``for a regular scalar (ie *non* vector) implementation (on a PPC G4
``processor); with a vector implementation of sigmoid2 (on the same
``processor), I actually got a whopping +250% speed increase...
`
So let me know what figures you get on a scalar x86 :-)
-- Olivier