[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-gnubg] Bug in sigmoid?

From: Joseph Heled
Subject: Re: [Bug-gnubg] Bug in sigmoid?
Date: Sat, 19 Apr 2003 09:01:59 +1200
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202

I am not sure about that. I have approximated exp(x) with 1+x, which is mathematically sound and has an o(x^2) error, which is nice in the [01] range the approximation is used. I think this is not the case for 1/(1+x)

Second, we need to re-train (we can't keep 2 nets around for practical reasons).

Third, I need to see numbers on how faster this is on non scalar, regular x86 machine.


Olivier Baur wrote:
Le jeudi, 17 avr 2003, à 22:04 Europe/Paris, Joseph Heled a écrit :


Can you please post the code for the new sigmoid? I did not understand the new formulation.

Sure, here it goes:

#define SIG_Q 10.0f    /* reciprocal of precision step */
#define SIG_MAX 100    /* half number of entries in lookup table */
/* note: the lookup table covers the interval [ -SIG_MAX/SIG_Q to +SIG_MAX/SIG_Q ] */

static float Sig[2*SIG_MAX+1];

static void ComputeSigTable (void)
    int i;

    for (i = -SIG_MAX; i < SIG_MAX+1; i++) {
        float x = (float) i / SIG_Q;
// Sig[SIG_MAX+i] = 1.0f / (1.0f + exp (x)); /* more accurate, but fails gnubgtest */ Sig[SIG_MAX+i] = sigmoid(x); /* use the current sigmoid function instead :-) */

static inline float sigmoid2 (float const x)
    float x2 = x * SIG_Q;
    int i = x2;

    if (i > - SIG_MAX) {
        if (i < SIG_MAX) {
            float a = Sig[i+SIG_MAX];
            float b = Sig[i+SIG_MAX+1];
            return a + (b - a) * (x2 - i);
return 1.0f / 19931.370438230298f; /* warning: this is 1.0f/(1.0f+exp(9.9f)) */
return 19930.370438230298f / 19931.370438230298f; /* warning: this is 1.0f/(1.0f+exp(-9.9f)) */

-- Olivier

Bug-gnubg mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]