[Bug-apl] [patch] float scanning broken for certain numeric locales & suggestions

[Robin Haberkorn]

Hi!

In Tokenizer::tokenize_real(), sscanf() is used to parse APL floating
point numbers in the input.
Unfortunately, sscanf()s behaviour regarding the decimal point is
locale-dependant, while APL's decimal point is always dot (.).

This means that if GNU APL was compiled with NLS, but your current
numeric locale (LC_NUMERIC) is not "C" (or any other locale with "." as
decimal point), the interpreter will not parse floating point numbers
correctly. In my case, LC_NUMERIC is "de_DE.UTF-8" by default, so APL
behaves as follows:

  $ apl --silent
        23.42
  23

I've attached a patch that sets LC_NUMERIC always to "C", so float are
scanned reliably. It applies to the SVN head (r107).
This is one of the easiest solutions that work as long as you don't
need any localized number scanning and printing.

Of course there are other possibilities:
  a) Force the user to set LC_NUMERIC properly. Doesn't sound like a
  good idea.
  b) Change the numeric locale temporarily while scanning floats.
  However there appears to be no way to do this in a thread-safe manner.
  c) Use C++ standard library streams, which allow you to set a locale
  on the stream. This would be thread-safe, but some people claim it's
  not very efficient.
  d) Roll your own float parsing algorithm.

An informative discussion of these issues can be found on Stack
Overflow:
http://stackoverflow.com/questions/13919817/sscanf-and-locales-how-does-one-really-parse-things-like-3-14

I also found other minor "issues" with GNU APL. I will just throw them
in here:
  * Printing of very long lines by the interpreter appears to be broken.
    E.g. try printing "⍳1000". There will be a long empty gap in the
    printout in the middle of the vector.
  * The Scan operator (\) is implemented according to the ISO specs
    using a quadratic time algorithm based on successive reductions. At
    least for some common function operands, scanning is possible in
    linear time however.
    Just compare the run times of "+/⍳10000" with "+\⍳10000". What do
    you think would be necessary to add support for linear time
    scanning?
  * I think that the APL expression 'ABC'⌷⍨¨⍳3 should yield the vector
    'A' 'B' 'C', while currently a RANK ERROR is reported. But that
    could be due to a lack of understanding -- I'm new to APL.
  * You should not commit generated files (autotools) into the SVN
    repository. Everyone who has to regenerate the autotools files for
    one reason or another is left with dozens of locally changed files
    and will face merge conflicts if you choose to commit your version
    of auto-generated files.
    I suggest removing all auto-generated files from the
    repository and setting the svn:ignore property on the parent
    directories to ignore these files instead. When you create source
    tar-balls with "make dist", automake is smart enough to include
    some of the auto-generated files like "configure" automatically.

Nevertheless GNU APL is a very useful tool! Well done!
I will release my first GNU APL-based project soon: a kind of simple
non-realtime sound synthesizer and manipulation toolkit.

best regards,
Robin

numeric_locale.patch
Description: Text Data