bug#13827: faulty range check in bytevector accessor

[Mark H Weaver]

Ian Price <address@hidden> writes:
> After some talk on #guile, Mark and I believe it comes down to the range
> check in INTEGER_ACCESSOR_PROLOGUE in bytevectors.c

Going a bit further: INTEGER_ACCESSOR_PROLOGUE uses 'scm_to_uint', which
I believe should fail for 2^32 on a 32-bit machine.  According to
numbers.h:430, 'scm_to_uint' should be an alias for 'scm_to_uint32',
which is defined in numbers.c:9277 and conv-uinteger.i.c:27.

It seems to me that it ought to be getting to conv-uinteger.i.c:50,
which calls 'mpz_fits_ulong_p'.  So maybe it's a bug in the version of
libgmp on Ian's machine, or perhaps I'm missing something.

I don't know whether it's possible to step through the code in
'conv-uinteger.i.c' using gdb.  If so, I'd like to see what happens.  If
not, I suspect the next step is to write some test programs in C and try
them on Ian's machine: first test 'scm_to_uint32', which should raise an
exception for 2^32.  If it doesn't then try testing 'mpz_fits_ulong_p'
directly and see if it's broken.

    Thanks,
      Mark