[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug #60923] Standard deviation returning NaN

From: Natáli Anzanello
Subject: [bug #60923] Standard deviation returning NaN
Date: Thu, 15 Jul 2021 21:44:55 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0


                 Summary: Standard deviation returning NaN
                 Project: GNU Astronomy Utilities
            Submitted by: ndanzanello
            Submitted on: Fri 16 Jul 2021 01:44:53 AM UTC
                Category: Libraries
                Severity: 3 - Normal
              Item Group: Output not reasonable
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any



The standard deviation calculation (within gal_statistics_std and
gal_statistics_mean_std) is returning a NaN value in some situations.

This is due to calculations involving floating numbers giving the square of
the average value greater than the average of the squares of the values, thus
making a square root of a negative number.


Reply to this item at:


  Message sent via Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]