Re: [Qemu-devel] [PATCH] vnc: tight: Fix crash after 2GB of output

[Roland Dreier]

On Fri, Mar 4, 2011 at 8:59 AM, Roland Dreier <address@hidden> wrote:
> Actually there is no problem with overflow of unsigned long.
> The C standard says that unsigned arithmetic is simply done
> modulo the size of the integer, so when total_out reaches
> 4GB, things will just wrap around (and the difference
> between "nearby" values will still be the correct, small
> value).  For example, if previous were (4GB - 5) and
> then total_out had 1000 added to it, total_out would
> end up as 995, and total_out - previous would be 1000.

Additionally, thinking about this further, I realize that
amusingly enough, the old code also works on 32-bit:
the bug occurred because when we put a value above
2GB in a (32-bit) int, it became a signed quantity,
which then became a gigantic value when promoted
back to an unsigned (64-bit) long, which causes the
subtraction to get the wrong value.  On 32-bit, the
promotion from signed 32-bit to unsigned 32-bit
doesn't lead to the wrong difference.

 - R.