[Octave-bug-tracker] [bug #57175] Incorrect output using rref() function on integer matrix

[Kai Torben Ohlhus]

Update of bug #57175 (project octave):

                Category:                    None => Octave Function        
                Priority:              5 - Normal => 3 - Low                
                  Status:                    None => Need Info              
                 Release:                   4.2.2 => 5.1.0                  
        Operating System:               GNU/Linux => Any                    

    _______________________________________________________

Follow-up Comment #1:

Thank you for the bug report.  Same can be observed with the current stable
Octave 5.1.0 release.

Please note, that in your example "A" is a double matrix, as it is the default
data type in Octave.  Use for example


A = int32 ([0,0,1,1,6;1,5,0,0,-28;0,0,-1,-25,-102;-1,-125,0,0,622])


to get a true integer matrix.

Unfortunately for your case, rref() is only defined for "double" and "single"
input.  This is true for Matlab R2019a as well, and the result there is the
same as in Octave.

To get a matrix rounded to integer like double values, please use


A = round (rref (A))


Does this satisfy your need?  I did not explicitly calculate your example by
hand.  But stepping through the function rref() with the debugger, I cannot
spot any wrong intermediate steps.

To resolve this bug report please clarify the following two questions:

- What supports your claim of your "correct answer"?
- Did you only expect a "true integer" output for your "integer stored inside
a double" input?


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57175>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/