Re: GCD bug?

[Ken Kahn]

Hi. Thanks.

I did some stepping through the code and found:

mp_limb_t b = mpn_submul_1 (up, vp, MIN (usize, vsize), q);

in bdivmod.c

that it crashes when usize=-1

The call stack was:
 __gmpn_bdivmod(unsigned long * qp=0x0012eafc, unsigned long *
up=0x0012eafc, long usize=-1, const unsigned long * vp=0x0012eb18, long
vsize=69, unsigned long d=4294965029) Line 98 C
  __gmpn_gcd(unsigned long * gp=0x0012efd4, unsigned long * up=0x0012e9e4,
long usize=69, unsigned long * vp=0x0012eb18, long vsize=69) Line 290 + 0x1d
C
  __gmpz_gcd(__mpz_struct * g=0x0012f664, const __mpz_struct * u=0x0012fc28,
const __mpz_struct * v=0x0012fc34) Line 138 + 0x3f C
  __gmpq_canonicalize(__mpq_struct * op=0x0012fc28) Line 41 + 0x14 C

By the way, I got the Visual Studio stuff from Dr. Brian Gladman
<address@hidden>. He says in his readme.txt:


The NASM assembler files are provided in the subdirectory gmp-4.1/mpn/x86i
and are translations of those in gas format in gmp-4.1/mpn/x86. I have only
translated the generic assembler files and those for the Intel PIII and PIV.

Thanks for your help with this.

Best,

-ken

----- Original Message -----
From: "Kevin Ryde" <address@hidden>
To: "Ken Kahn" <address@hidden>
Cc: <address@hidden>
Sent: Monday, March 31, 2003 4:37 PM
Subject: Re: GCD bug?


> "Ken Kahn" <address@hidden> writes:
> >
> > I get an access violation from __gmpn_submul_1()+0x56
>
> Seems ok for me on an i386 gnu system.  Does a backtrace say more?
>
> You might like to check the return value from each mpz_set_str, in
> case something has gone wrong in the parsing.
>
> --
> All followups to address@hidden please.
>