Re: Character folding in the pretest

[Elias Mårtenson]

On 5 Feb 2016 1:06 a.m., "Werner LEMBERG" <address@hidden> wrote:
>
> This naturally leads to a possible user option: Having `optical'
> matches or not, where `optical' means `base character plus diacritic
> and/or slight modifications', e.g., o → ø → ö etc., etc.

I think this statement shows how easy it is to introduce cultural bias, although the fact that your name sounds German suggests that personal preference is involved.

How do you even define "optical similarities"? Should l and I compare the same under this definition? They certainly looks similar. What about p and q? They look like mirror images of each other. What about z and s? They even sound similar. To a Swedish speaker there are zero similarities between a, ä and å. They are, in fact, just as different as a and z are to an English speaker. I really cannot emphasise this enough, and reading this thread tells me that it needs to be emphasised even more.

As someone who lives in an English speaking country and using English keyboards, while still working with documents in various languages, I see first-hand the need to have ways of searching for characters that I can't easily type on my keyboard, but this issue is orthogonal to that of character equivalence. The conflating of these two issues are, in my opinion, the root cause of many of the disagreements in this thread.

My personal preference is that the expected behaviour of searches is more related to the locale of the user, rather than that of the document being searched. In other words, as a non-Spanish speaker, I'd expect to be able to find ñ when searching for n, even if the document I'm searching in is in Spanish. There are definitely an infinite number of counter-examples to this (enough to keep this thread going for another 100 messages, I'm sure), but at least there is reason to consider making the default based on the locale of the user.