[Octave-bug-tracker] [bug #33503] null.m: tolerance for zero in basis elements

[Olaf Till]

URL:
  <http://savannah.gnu.org/bugs/?33503>

                 Summary: null.m: tolerance for zero in basis elements
                 Project: GNU Octave
            Submitted by: i7tiol
            Submitted on: Thu 09 Jun 2011 05:03:26 PM GMT
                Category: None
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: Olaf Till
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: GNU/Linux

    _______________________________________________________

Details:

Octave version: 3.5.0+, stable branch, release 5bf8af73fc34 (2011-06-01)

Short description: tolerance for setting basis elements to zero possibly too
high.

Long description: Unfortunately I can't theoretically derive a suitable value
for this tolerance. But I had a practical problem in which for theoretical
reasons a certain element of the basis of the null-space of some matrix should
be zero. Due to very slight differences in this matrix (accuracy of
calculation) between two systems with Octave, the said element was returned by
null() as zero on one system and not as zero on the other system (same null()
version, in particular same tolerance used: eps). A larger tolerance "tol" is
used by null() for assuming elements of S (from svd()) to be zero. If this
tolerance "tol" were also used for setting elements of the returned basis to
zero, I would get the expected result of zero on both systems (the order of
magnitude of "tol" would be just enough for that).

So I suggest that null() should use the calculated tolerance "tol" also for
setting basis elements to zero.





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?33503>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/