bug-gnubg
[Top][All Lists]

## Re: [Bug-gnubg] Bug in sigmoid?

 From: Nis Subject: Re: [Bug-gnubg] Bug in sigmoid? Date: Thu, 17 Apr 2003 18:28:32 +0200

--On Thursday, April 17, 2003 15:33 +0200 Olivier Baur <address@hidden> wrote:
```
```
```I think I've found a bug in sigmoid (neuralnet.c), but I'm not sure about
its impact on the evaluation function...
```
```
```
I don't understand the role of this function fully, but I would guess that the actual function doesn't matter much, as long as it is relatively smooth and increasing.
```

```
```Let's call S the real sigmoid function: S(x) = 1 / ( 1 + e^x)
It seems that sigmoid(x) will return a good approximation of S(x) for
-10.0 < x < 10.0 (less than +/-.01% error), but then it returns S(9.9)
for x >= 10.0 (instead of S(10.0)) and S(-9.9) for x <= -10.0 (instead of
S(-10.0)). sigmoid is not even monotonic!
```
```
```
The big question for me is: Does it matter if we change the function - for instance to get it closer to S(x).
```
```
More specifically: Is the current sigmoid function "imprinted" on the synapses of the current nets?
```
```
```By the way, I found a simple way of optimising the current sigmoid
function: instead of having a lookup table holding pre-computed values of
exp(X) and then returning sigmoid(x) = sigmoid(X+dx) =
1/(1+exp(X)(1+dx)), why not have a lookup table holding precomputed
values of S(X) and return sigmoid(x) = sigmoid(X+dx) =
S(X)+dx.(S(X+1)-S(X))? The time consumming operations here are the
lookups and the reciprocal (1/x) operations. With the second method, you
trade one reciprocal and one lookup for two lookups; and since in the
latter case the second lookup will probably already be in the processor
cache (since S(X+1) follows S(X) in memory), you end up doing mostly one
lookup and no more reciprocal. On my machine, it gave me a +60% speed
increase in sigmoid.
```
```
Beautiful. Did you compare the precision of the results?

--
Nis Jorgensen
Greenpeace
Amsterdam

```

reply via email to