pspp-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] New module to perform decimal floating point arithmetic for


From: Ben Pfaff
Subject: Re: [PATCH] New module to perform decimal floating point arithmetic for charts.
Date: Tue, 13 Jan 2015 21:24:37 -0800
User-agent: Mutt/1.5.23 (2014-03-12)

On Tue, Jan 13, 2015 at 07:03:17AM +0100, John Darrington wrote:
> This change adds a small module to perform floating point arithmetic
> with a decimal base, and uses it to calculate the graticule marks for
> charts.
> 
> Rationale:  Graticule marks want to be displayed in decimal.  However
> not all decimal values can be represented in a double precision
> floating point binary.  This approach pushes the precision loss from
> the value of the mark, to the position on the chart - we don't
> particularly care if a tick mark is a fraction of a pixel out of place,
> but we do care if 4.0 is displayed as 3.9999999999

Can you say a few words about the assumptions?  At first glance, some of
the functions seem to suffer from catastrophic loss of precision upon
integer overflow, e.g. doesn't that happen in decimal_multiply() if the
intermediate result is greater than ORD_MAX?  (Is some kind of care
taken to make sure that this cannot happen?)



reply via email to

[Prev in Thread] Current Thread [Next in Thread]