Re: On language-dependent defaults for character-folding

[Juri Linkov]

> But the most basic issue is that any significant development in these
> directions require to re-implement the feature on the C level, and use
> char-tables for folding, like we do with case-mapping.  So until
> someone steps forward for the job, all we can do is small corrections
> to the existing implementation.

Do I understand correctly that essentially what is necessary to do on the
C level is to extend char-tables with character insertions and deletions,
so in addition to canonical equivalence mappings (like are used for the
existing case-mappings) char-tables should also support matching of
multi-character additions (like combining accents in the search
string) and deletions (like combining accents from the search string
missing in the search text)?

> For example, the default state of character-folding might depend on
> the locale's language -- we could turn it off by default for languages
> whose users expressed dissatisfaction with the feature.  We could also
> augment the regular expressions created for folding the search string
> by filtering out variants that users of a particular language don't
> want.  If people think these ideas will make more users happy, we can
> work on that.

It seems two user variables are necessary for customization:

1. inclusive folding groups that will include by default such pairs
   as o - ø, l - ł added to the Unicode decomposition-based rules,
   and allow the users to add more rules;

2. exclusive folding groups to exclude locale/language-dependent rules from
   the default mappings above, e.g. removing n - ñ for the "es" locale.