[Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16

[Mayeul Cantan]

Follow-up Comment #2, bug #63089 (project octave):

Affected file is
https://hg.savannah.gnu.org/hgweb/octave/file/b542b88ad3b6/scripts/strings/dec2bin.m

I was going to write a comment to the effect of "I won't have time to work on
this before ~1 month", but I got nerd-sniped (I really should have worked on
something else instead).

Anyway, I'm not too proud of how I implemented compatibility with arrays (and
growing strings to the same lenghts), but I think the file I'm attaching is
feature-complete.

If you want to refactor, the gist is:

% Perform the base conversion on:
i = uint64(-1*(int64(i)+1));
out = dec2base(i, 2, len)
% Then invert the output:
out = char((out=='0') + '0');


Things I'm not sure of:
* How Matlab handles length extension. I implemented what made more sense to
me. See comments in tests.
* Apparently this was supposed not to work on floating point values? From my
testing it does work on them, and I see no reason not to (I added a check to
error after fmaxint(i)).
* If there is a better way to concatenate values with the correct sign
extension

Ideally, I'd have wanted (u)int<n> to default to <n> output digits, especially
if <n> isn't default. But I can live with the attached implementation.

(file #53739)

    _______________________________________________________

Additional Item Attachment:

File name: dec2bin.m                      Size:7 KB
    <https://file.savannah.gnu.org/file/dec2bin.m?file_id=53739>



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63089>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/