Re: [Bug-apl] number precision in saved workspaces

[Xiao-Yong Jin]

Somehow, 1⎕TF is slightly slower (see the 2nd and the 3rd rows in the following)

      tio¨1E5×⍳5
  ⍕    619     ⍕    1306     ⍕    2007     ⍕    2605     ⍕    3425  
  1⎕tf 230     1⎕tf  461     1⎕tf  819     1⎕tf 1094     1⎕tf 1383  
  ⍎    158     ⍎     327     ⍎     562     ⍎     695     ⍎     859  
      ∇tio[⎕]∇
    ∇
[0]   r←tio n;t;x;z;s;⎕PP
[1]   ⎕PP←17
[2]   x←1E¯18×?n⍴1E18
[3]  ⍝ t←⎕ai◊⊣1⎕tf'x'◊r←(⊂'1⎕tf'),2⌷⎕ai-t    ⍝ this is very slow
[4]   t←⎕ai◊s←⍕x◊r←(⊂'⍕'),2⌷⎕ai-t
[5]   t←⎕ai◊⊣1⎕tf'Nz 1 ',(⍕n),' ',s◊r←r,[.5](⊂'1⎕tf'),2⌷⎕ai-t
[6]   t←⎕ai◊⊣⍎s◊r←r,[1](⊂'⍎'),2⌷⎕ai-t
[7]   (x≢z)(x≢⍎s)/'E:x≢z' 'E:x≢⍎s'
    ∇

⍎ in gnu apl is 4 times faster than dyalog now.  I think it’s good enough.  (⍕ 
is 4 times slower than dyalog though.)

I just found out that my script being much slower is actually related to the 
new macro system having bad scaling behavior.  I’ll send a separate email about 
it.

Best,
Xiao-Yong


> On Jun 16, 2016, at 10:02 AM, Juergen Sauermann <address@hidden> wrote:
> 
> Hi Xiao-Yong,
> 
> the problem with the *scanf() *(and also with the *printf()*) functions is 
> that text of numbers
> is different from APL, for example *-* vs. *¯*, complex numbers, etc.
> 
> I therefore believe it would be more useful to have a *⎕CR* variant that 
> avoids full
> parsing and execution of its argument (which *⍎* does). Actually *⎕TF* is 
> already doing this
> and should be much faster than *⍎*, for example:
> 
> *      ⍝ create numeric variable**
> **      R←1.1 2J2 4 1R2.2 ¯5 6E¯2 **
> ****
> **      ⍝ convert variable into text (to be written to a file)**
> **      ⎕←TEXT←1 ⎕TF 'R'**
> **NR 1 6 1.1000000000000001 2J2 4 
> ¯5.8850111725534582E¯1J8.0849640381959009E¯1 ¯5 **6E¯2**
> ****
> **      )ERASE R*
> 
> ***      ⍝ read text back from file and re-create variable**
> **      1 ⎕TF TEXT**
> 
> ****      R**
> **1.1 2J2 4 ¯0.5885011173J0.8084964038 ¯5 0.06**
> *
> The *1 ⎕TF* record format is fairly simple (record type (N) followed by the 
> variable name (R in the example) , rank, and shape of the value).
> The inverse *1 ⎕TF* tokenizes the string but does, unlike *⍎*, neither parse 
> nor execute it.
> 
> Please use *SVN 745* since I have fixed some *1 ⎕TF* bugs and improved its 
> error reporting.
> 
> /// Jürgen
> 
> 
> On 06/16/2016 12:35 AM, Xiao-Yong Jin wrote:
>> Hi Jürgen,
>> 
>> Just realized my script got much slower (⍕ on a few million numbers from a 
>> text file) with these changes.  Since you have printf in ⎕FIO, can you 
>> please add scanf, fscanf, and strto* functions to ⎕FIO, too?
>> 
>> In addition, the tokenizer seems to be confused with exponents.
>> 
>>       1E9
>> 1000000000
>>       1E10
>> 1E10
>>       1E9×1E9
>> 1E18
>>       ?1E10
>> 3942470777
>>       2?1E10
>> DOMAIN ERROR
>>       2?1E10
>>       ^^
>> 
>> Wouldn't anything below 9E18 be parsed as an integer?
>> 
>> Best,
>> Xiao-Yong
>> 
>>> On Jun 13, 2016, at 9:24 AM, Xiao-Yong Jin <address@hidden> wrote:
>>> 
>>> Yes, they work.  Thanks.
>>> 
>>>> On Jun 11, 2016, at 9:29 AM, Juergen Sauermann <address@hidden> wrote:
>>>> 
>>>> Hi Xiao-Yong,
>>>> 
>>>> thanks, fixed in SVN 740. I haven't checked )DUMP and )OUT but I suppose
>>>> they work now since the fault was in the tokenizer (which is also used by
>>>> )LOAD and )IN).
>>>> 
>>>> /// Jürgen
>>>> 
>>>> 
>>>> On 06/10/2016 10:34 PM, Xiao-Yong Jin wrote:
>>>>> Hi Jürgen,
>>>>> 
>>>>> Thanks.  I found more floating point reproduction issues.
>>>>> 
>>>>>      ⎕PP←17
>>>>>      x-y←x←○3 4
>>>>> 0 0
>>>>>      x-⍎⍕x
>>>>> 0 1.7763568394002505E¯15
>>>>>      x-¯14⎕CR 14⎕CR x
>>>>> 0 0
>>>>>      2⎕TF 2⎕TF ‘y'
>>>>> y
>>>>>      y
>>>>> 9.4247779607693793 12.566370614359172
>>>>>      x
>>>>> 9.4247779607693793 12.566370614359172
>>>>>      x-○3 4
>>>>> 0 0
>>>>>      y-○3 4
>>>>> 0 ¯1.7763568394002505E¯15
>>>>>      x-y
>>>>> 0 1.7763568394002505E¯15
>>>>>      9.4247779607693793 12.566370614359172-○3 4
>>>>> 0 ¯1.7763568394002505E¯15
>>>>> 
>>>>> 
>>>>> The exact representation by 14⎕CR works great (as in one of the ISO CF 
>>>>> implementations), but both ⎕TF and ⍎⍕ fail to reproduce the exact number.
>>>>> 
>>>>> Since you last change, the )save )load does keep the exact values, and I 
>>>>> looked at the saved workspace file, the numbers are
>>>>> 146   <Ravel vid="1" cells="⁴9.4247779607693793⁴12.566370614359171"/>
>>>>> 147   <Ravel vid="2" cells="⁴9.4247779607693793⁴12.566370614359172"/>
>>>>> 185     <Symbol name="x" stack-size="1">
>>>>> 186       <Variable vid="2"/>
>>>>> 187     </Symbol>
>>>>> 188
>>>>> 189     <Symbol name="y" stack-size="1">
>>>>> 190       <Variable vid="1"/>
>>>>> 191     </Symbol>
>>>>> Here, the last digit of y is 1 instead of 2, as shown in the REPL.
>>>>> 
>>>>> So it seems the routine used to format numbers has some problem with the 
>>>>> last digits, even if with ⎕PP=17?  Or it’s just the reader has some 
>>>>> problem?
>>>>> 
>>>>> In addition, )dump and )out still changes the floating point number, but 
>>>>> perhaps that is the same issue as in ⎕TF and ⍎⍕?
>>>>> 
>>>>> 
>>>>>> On Jun 7, 2016, at 3:29 PM, Juergen Sauermann <address@hidden>
>>>>>> wrote:
>>>>>> 
>>>>>> Hi Xiao-Yong,
>>>>>> 
>>>>>> OK - SVN 739.
>>>>>> 
>>>>>> /// Jürgen
>>>>>> 
>>>>>> 
>>>>>> On 06/07/2016 08:16 PM, Xiao-Yong Jin wrote:
>>>>>> 
>>>>>>> Hi Jürgen,
>>>>>>> 
>>>>>>> Thanks, though the current behavior still does not give the exact 
>>>>>>> number bit by bit.  Can you change it to %.17g, or 
>>>>>>> std::numeric_limits<double>::max_digits10?
>>>>>>> 
>>>>>>> In addition, is it OK to raise the maximum allowed ⎕PP to 17?
>>>>>>> 
>>>>>>> Best,
>>>>>>> Xiao-Yong
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On Jun 7, 2016, at 12:37 PM, Juergen Sauermann <address@hidden>
>>>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi Xiao-Yong,
>>>>>>>> 
>>>>>>>> thanks, fixed in SVN 738.
>>>>>>>> 
>>>>>>>> /// Jürgen
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 06/07/2016 08:13 AM, Xiao-Yong Jin wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Hello,
>>>>>>>>> 
>>>>>>>>> Is the following behavior intended?  Note the x value after load.
>>>>>>>>> 
>>>>>>>>>      )clear
>>>>>>>>> CLEAR WS
>>>>>>>>>      x←.12345678901234567890
>>>>>>>>>      ⎕pp←16
>>>>>>>>>      x
>>>>>>>>> 0.1234567890123457
>>>>>>>>>      )wsid tmp
>>>>>>>>> WAS CLEAR WS
>>>>>>>>>      )save tmp
>>>>>>>>> 2016-06-07  01:09:15 (GMT-5)
>>>>>>>>>      )clear
>>>>>>>>> CLEAR WS
>>>>>>>>>      )load tmp
>>>>>>>>> SAVED 2016-06-07 01:09:15 (GMT-5)
>>>>>>>>>      x
>>>>>>>>> 0.123457
>>>>>>>>>      ⎕pp
>>>>>>>>> 16
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>> 
>