[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CSV parsing and other issues (Re: LC_NUMERIC)

From: Maxim Nikulin
Subject: Re: CSV parsing and other issues (Re: LC_NUMERIC)
Date: Tue, 8 Jun 2021 23:35:51 +0700
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1

On 08/06/2021 09:35, Eli Zaretskii wrote:
> From: Boruch Baum
>> No? If an Emacs user has two buffers in two separate languages, the
>> buffer-local settings aren't / won't be respected?
> First, language is different from locale.  And second, we don't even
> have a buffer-local notion of language yet.

Certainly locale is more precise than just language since it includes region and other variants, moreover it can be granularly tuned (date, numbers, sorting can be adjusted independently), but I still think that all these properties can be sometimes broadly referred to as language.

Does not we discuss a feature request? Low level functions can accept explicit locale. Higher level API can obtain it implicitly from buffer-local variables and global locale. For example the LOCALE argument of `string-collate-lessp' is optional one. I can even anticipate that locale may be stored in text properties some times. A random message from recent "About multilingual documents" thread at emacs-orgmode mail list:

At first basic functionality may be implemented. The problem is to choose extensible API.

On 07/06/2021 06:36, Boruch Baum wrote:
> I get 'nil' when I check for any related
> environment variable using function `getenv'

Do not confuse setlocale and setenv. setenv affects
later calls to setlocale (with NULL as locale argument)
and child processes. setlocale deals with current
processes it can take into account or override values
of environment variables. setlocale is not exposed to elisp.

The single quote format (for the thousands separator) can be expected
to produce a result always for all conditions of locale, while I
expect most locale cases won't produce any special output for the
upper-case I format option.

I still think that "'" and "I" formats are tightly bound. Grouping style is locale-dependent. So representation of digits is just another property of locale.

LC_NUMERIC=C.UTF-8 /usr/bin/printf "%'d\n" 1234567890
LC_NUMERIC=en_US.UTF-8 /usr/bin/printf "%'d\n" 1234567890
LC_NUMERIC=es_ES.UTF-8 /usr/bin/printf "%'d\n" 1234567890
LC_NUMERIC=ru_RU.UTF-8 /usr/bin/printf "%'d\n" 1234567890
1 234 567 890

Even group size is not always 3

: LC_NUMERIC=en_IN.UTF-8 /usr/bin/printf "%'d\n" 1234567890
: 1,23,45,67,890

"India uses thousands/lakh/crore separators"

I just have realized that nl_langinfo(3) (and nl_langinfo_l(3) as well) from libc accepts RADIXCHAR (decimal dot) and THOUSEP (group separator) arguments. They are good candidates for `locale-info' extension.

On 05/06/2021 02:17, Eli Zaretskii wrote:
>> From: Maxim Nikulin Date: Fri, 4 Jun 2021 23:31:13 +0700
>> On linux I see that Emacs is linked with ICU
> It isn't.  It's either HarfBuzz or maybe libc that pulls in the ICU
> library.  Emacs doesn't use it directly.

Actually Qt links my example with other libraries from ICU. My point was that since Emacs anyway (indirectly) links with this library, the dependency may be not so heavy. My personal requirements for number formatting were quite modest so far, I expect that other languages (CJK, right-to-left scripts, etc.) may require quite special treatment, so implementation in Emacs (and further maintenance) may require a lot of work. At least API of ICU should be studied to get some inspiration what features will be necessary for users from other regions.

E.g. I was completely unaware that negative sign may be represented by parenthesis (JavaScript, may be executed in browser developer tools)

new Intl.NumberFormat('en-GB', {
  style: 'currency',
  currency: 'USD',
  currencySign: 'accounting',
  signDisplay: 'always'


I do not know if Intl API is really convenient. I see that there is no
direct way to get decimal separator. However it can serve as
another source for inspiration.

I expect enough surprises and unexpected "discoveries" during implementation of better locale support. That is why I would consider adapting some more or less established API for this purpose.


On 07/06/2021 06:36, Boruch Baum wrote:
and in practice I need to temporarily manually use function setenv to
set LC_COLLATE=C in order to offer several sorting options in package

Ideally you should avoid this and use envp argument of execve(2) system call. Otherwise it could interfere with other packages, especially if threads are involved. Unsure that Emacs currently provides such API option.

it's temporarily setting a shell environment and having the external
ls program perform the sort for emacs-core dired.

I am unsure if "ls" may be reliably used at all. File names may have e.g. newlines, various control characters, part that looks rather similar to ls output. I am not familiar with dired internals. At first by intention was to create an issue for diredc but skimming though its code I did not found direct "ls" invocation. Some problems with ls:
Bash Pitfalls: item #1

reply via email to

[Prev in Thread] Current Thread [Next in Thread]