[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Difference double/single precision

From: James Sherman Jr.
Subject: Re: Difference double/single precision
Date: Wed, 21 Sep 2011 19:09:01 -0400

On Wed, Sep 21, 2011 at 6:53 PM, Schirmacher, Rolf <address@hidden> wrote:
with the 3.2.4 windows (mingw) binary, I get the following results:
octave.exe> format long
octave.exe> 1.0/3
ans =  0.333333333333333
octave.exe> single(1.0/3)
ans =  0.333333343267441
octave.exe> 1.0/3 - single(1.0/3)
ans = 0
I am wondering about the difference being 0.
In addition, how could I calculate the difference of 1.0/3 as single and double?
Thanks for any hint,

I get tripped up on these types of problems myself.  The key thing to remember for me is that when you are doing operations with different classes of variables, you need to check what octave does to be able to (in this case) subtract them.  Octave in this case, I believe, of a double minus a single, will convert the double to a single, then subtract them and the result will be of the single class. (Hence why the result is 0.) To avoid this, you could convert your single class number, back into a double before subtracting it.  Like this:

1.0/3 - double(single(1.0/3))

Should give you the "rounding" error of converting to single precision.  I don't have octave on hand right now to double check this however.

Hope this helps.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]