[Octave-bug-tracker] [bug #63785] float compared to 0 in __ichol__.cc

[Rik]

Follow-up Comment #1, bug #63785 (project octave):

I think in general you are right that we shouldn't be comparing exact values
(derived from pure math) with calculated values made on computers with finite
precision.

My only question is the actual tolerance itself.  The value

```
eps * real (pivot)
```

works.  But, is there a theoretical justification for that number given the
ichol algorithm?  I don't know because I don't know this math well enough to
say.

If we don't have a theoretical basis, another way would be to try and
determine the tolerance empirically.  Have the system print the value of

```cpp
std::fabs(pivot.imag ())
std::fabs(pivot.real ())
```

and then try it on a few failing matrices.  We then set the comparison
threshold to something that works for these matrices and see if there are
further bug reports.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63785>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/