[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negativ
From: |
Arun Giridhar |
Subject: |
[Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16 |
Date: |
Sat, 24 Sep 2022 11:17:39 -0400 (EDT) |
Follow-up Comment #5, bug #63089 (project octave):
I am attaching a WIP patch for testing. It is vectorized (no loops) and can
accommodate negative numbers below flintmax to the full range of int64 for
signed and uint64 for unsigned. It passes all BISTs except for these:
dec2bin (-2^31 -1)
dec2bin (-2^52)
error <negative inputs cannot be less than> dec2bin (- flintmax ())
In the first two cases, the expected result was a 53-bit string but the
patched version returns a 64-bit string with more leading 1s, which on
reflection I think is an improvement since the other conversions of negative
numbers when length is not specified return 8, 16, or 32 bits, so returning a
bit string with 64 bits is more consistent than 53 bits.
The last failing BIST is because the patched version *can* handle negative
numbers below flintmax, so the expected error does not happen.
The trimming-to-length part is subjective for negative numbers, because one
can either return the minimum number of bits plus a leading "1", or one can
return a fixed power-of-2 number of bits with several leading "1"s. Both
choices can be defended. The patch at present does the second case, which is
similar what C++ does when casting signed to unsigned, returning e.g. 32 bits
or 64 bits but not something in between:
int32_t c = -123456789;
int64_t d = -123456789;
std::cout << uint32_t (c) << ' ' << uint64_t (d) << '\n';
Please test against Matlab for the following cases:
dec2bin (-3) # should return 8 bits
dec2bin (-4200) # should return 16 bits
dec2bin (int16(-4200), 5) # should hopefully still return 16 bits, not just
13 bits
dec2bin (intmin ("int64")) # should return one 1 followed by 63 zeroes.
(file #53747)
_______________________________________________________
Additional Item Attachment:
File name: dec2bin.patch Size:2 KB
<https://file.savannah.gnu.org/file/dec2bin.patch?file_id=53747>
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?63089>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Mayeul Cantan, 2022/09/21
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Nicholas Jankowski, 2022/09/21
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Mayeul Cantan, 2022/09/21
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Markus Mützel, 2022/09/23
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Nicholas Jankowski, 2022/09/23
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16,
Arun Giridhar <=
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Arun Giridhar, 2022/09/24
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Arun Giridhar, 2022/09/24
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, A.R. Burgers, 2022/09/24
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Arun Giridhar, 2022/09/25
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Arun Giridhar, 2022/09/28
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Markus Mützel, 2022/09/29
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Arun Giridhar, 2022/09/29
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Arun Giridhar, 2022/09/29
- [Octave-bug-tracker] [bug #63089] dec2bin gives wrong answer for negative int16, Markus Mützel, 2022/09/29