help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: variable precision arithmetic (vpa) errant results in windows 10, o


From: Colin Macdonald
Subject: Re: variable precision arithmetic (vpa) errant results in windows 10, octave-4.0.1
Date: Wed, 22 Jun 2016 08:31:54 -0700
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1

On 21/06/16 21:30, reik red wrote:
As you can see, the number "1/3" is inaccurately represented when digits

I haven't read all that but have you read "help vpa"?

There are ok:

vpa("1/3", 64)
vpa(sym(1)/3, 64)
vpa(1, 64)

Whereas vpa(1/3, 64) first creates the a double precision floating point number near 1/3 and then converts it to have 64 digits, which must be lossy.

Colin



reply via email to

[Prev in Thread] Current Thread [Next in Thread]