[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
numeric error in strtonum()
From: |
Hans-Peter Dukek |
Subject: |
numeric error in strtonum() |
Date: |
Fri, 29 Nov 2024 10:46:54 +0100 |
Dear gawk-team,
when converting nanosecond epoch timestamps (i.e. large hex numbers in 64 bit
entities) with the strtonum function, I discovered by coincidence that the
results differed from bash and dc calculations.
Version and examples can be found below.
Kind regards
H-P
hp@bullseye-server:/tmp$ uname -a
Linux bullseye-server.local.dukek.net 5.10.0-33-amd64 #1 SMP Debian 5.10.226-1
(2024-10-03) x86_64 GNU/Linux
hp@bullseye-server:/tmp$ gawk --version
GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0, GNU MP 6.2.1)
Copyright (C) 1989, 1991-2020 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
hp@bullseye-server:/tmp$ echo 0x180C66CE7576F79B | awk '{print strtonum($1)}'
1732872993550825472
hp@bullseye-server:/tmp$ echo $((0x180C66CE7576F79B))
1732872993550825371
hp@bullseye-server:/tmp$ echo "ibase=16 ; 180C66CE7576F79B" | bc
1732872993550825371
hp@bullseye-server:/tmp$ echo 0x180C66E84D8BC315 | awk '{print strtonum($1)}'
1732873104550249216
hp@bullseye-server:/tmp$ echo $((0x180C66E84D8BC315))
1732873104550249237
hp@bullseye-server:/tmp$ echo "ibase=16 ; 180C66E84D8BC315" | bc
1732873104550249237
- numeric error in strtonum(),
Hans-Peter Dukek <=