dotgnu-pnet
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Pnet-developers] Fixing bugs in floating-point formatting


From: Marcus
Subject: [Pnet-developers] Fixing bugs in floating-point formatting
Date: Sun, 18 Dec 2005 19:00:24 -0600
User-agent: KMail/1.9.1

I am trying to fix some bugs the output formatting of floating point types, 
specifically System.Single and System.Double. (System.Decimal seems to be a 
bit different.) I have found two problems.

One problem is that the output string always has one too many digits. I fixed 
that by changing the lines in FormatFloat() to

        if (sb.Length > precision + exponent)
                {
                        sb.Remove(precision+exponent, 
sb.Length-(precision+exponent));
                }

Another problem is that the output digits are always truncated (by removing 
characters from the StringBuilder). In MS, the digits are rounded instead 
(and not using "banker's rounding) of truncated. 

I am also confused about the role of the local variable "exponent" because it 
always seems to have the valve 0 for the cases I tried.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]