[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH] vnc: tight: Fix crash after 2GB of output

From: Roland Dreier
Subject: Re: [Qemu-devel] [PATCH] vnc: tight: Fix crash after 2GB of output
Date: Fri, 4 Mar 2011 08:59:59 -0800

On Thu, Mar 3, 2011 at 11:34 PM, Michael Tokarev <address@hidden> wrote:
>> The fix for this is simple: keep previous_out as a uLong too, which
>> avoids any problems with sign conversion or truncation.
> This looks wrong to me.  On 32bit x86 uLong is 32bits.  Yes
> it's unsigned there, but it's still 32bits.  And sure thing
> it _will_ overflow again, not at 2Gb but now at 4Gb, and the
> next total_out will start from zero again, so that now bytes
> will be a large integer again because "next" (new total_out)
> will be less than "current" (previous_out).

Actually there is no problem with overflow of unsigned long.
The C standard says that unsigned arithmetic is simply done
modulo the size of the integer, so when total_out reaches
4GB, things will just wrap around (and the difference
between "nearby" values will still be the correct, small
value).  For example, if previous were (4GB - 5) and
then total_out had 1000 added to it, total_out would
end up as 995, and total_out - previous would be 1000.

In other words I'm pretty sure my fix works on 32 bits too.

However your fix using avail_out is probably better anyway,
and your latest patch fixes both instances of the bug, so
we might as well go with that.

 - R.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]