Re: [Bug-apl] Fuzzy Floor and Ceiling

[Juergen Sauermann]

Hi Mike,

the problem with relative ⎕CT is that when the N gets large, say > ÷⎕CT then N×⎕CT becomes > 1 so the
interval around N where you would round up (for ⌊) contains one integer or even several. The ISO intention
seems to be that only numbers that are slightly smaller than N but very close  to N should be handled like N
and not like N-1. Mathematically I see no good reason why this interval should grow when N gets larger.

Another problem is that the IBM APL implementation differs from the IBM APL2 description,
which says (APL2 language reference manual page 133):

Z←R For real numbers, yields the largest integer that does not exceed R (within the comparison tolerance).

That is essentially what ISO says and what GNU APL has implemented. The observed behavior
of the IBM APL2 implementation seems to be (within R times the comparison tolerance),
which is probably unintentional.

/// Jürgen

On 08/14/2015 07:34 PM, Mike Duvos wrote:

Hi Juergen,

You write...

"Summary: For ⌈ and ⌊ GNU APL follows the ISO standard while IBM APL2 does not."

While this may be the case, I would argue that the APL2 approach is much more numerically robust than the ISO approach. 

Within the range of contiguous integers exactly representable as 64 bit floats,  only 2049 of them will have a different value after having  ⎕CT=1E¯13 added to them.  For the rest of them, an additional 13 decimal digits of mantissa bits won't fit into the float, and fudging by an absolute ⎕CT will be a no-op.

It is hard to argue that floor and ceiling are "tolerant" if they only do a fuzzy comparison on a few numbers, and a regular comparison on all the rest.

In this case, it would seem that doing it the APL2 way, and not the ISO way, would be the prudent choice.

Regards,

Mike