bug-gnubg
[Top][All Lists]

## Re: [Bug-gnubg] Bug in sigmoid?

 From: Joseph Heled Subject: Re: [Bug-gnubg] Bug in sigmoid? Date: Sat, 19 Apr 2003 09:01:59 +1200 User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202

```
```
I am not sure about that. I have approximated exp(x) with 1+x, which is mathematically sound and has an o(x^2) error, which is nice in the  range the approximation is used. I think this is not the case for 1/(1+x)
```
```
Second, we need to re-train (we can't keep 2 nets around for practical reasons).
```
```
Third, I need to see numbers on how faster this is on non scalar, regular x86 machine.
```
-Joseph

Olivier Baur wrote:
```
```Le jeudi, 17 avr 2003, à 22:04 Europe/Paris, Joseph Heled a écrit :

```
```
Hi,

```
Can you please post the code for the new sigmoid? I did not understand the new formulation.
```
```
```
Sure, here it goes:

#define SIG_Q 10.0f    /* reciprocal of precision step */
#define SIG_MAX 100    /* half number of entries in lookup table */
```
/* note: the lookup table covers the interval [ -SIG_MAX/SIG_Q to +SIG_MAX/SIG_Q ] */
```
static float Sig[2*SIG_MAX+1];

static void ComputeSigTable (void)
{
int i;

for (i = -SIG_MAX; i < SIG_MAX+1; i++) {
float x = (float) i / SIG_Q;
```
// Sig[SIG_MAX+i] = 1.0f / (1.0f + exp (x)); /* more accurate, but fails gnubgtest */ Sig[SIG_MAX+i] = sigmoid(x); /* use the current sigmoid function instead :-) */
```    }
}

static inline float sigmoid2 (float const x)
{
float x2 = x * SIG_Q;
int i = x2;

if (i > - SIG_MAX) {
if (i < SIG_MAX) {
float a = Sig[i+SIG_MAX];
float b = Sig[i+SIG_MAX+1];
return a + (b - a) * (x2 - i);
}
else
```
return 1.0f / 19931.370438230298f; /* warning: this is 1.0f/(1.0f+exp(9.9f)) */
```    }
else
```
return 19930.370438230298f / 19931.370438230298f; /* warning: this is 1.0f/(1.0f+exp(-9.9f)) */
```}

-- Olivier

_______________________________________________
Bug-gnubg mailing list
```