[emacs-bidi] Re: RTL support

[Eli Zaretskii]

> Date: Fri, 25 Nov 2005 11:24:19 -0600
> From: Gregg Reynolds <address@hidden>
> CC: address@hidden, Eli Zaretskii <address@hidden>
> 
> I guess the point is that we can get there in stages.  First you
> implement RTL layout, then shaping, then bidi.  That way we have
> *usable* software without having to wait for bidi support, and
> eventually we do have full bidi support.

>From what I understand, users of RTL languages want almost full bidi
support, they won't settle for RTL display alone.

> Vim provides the model

I never saw anyone use Vim for Hebrew text---that I can tell you.
Hebrew text without reordering is going backwards to the age of
typewriters when secretaries needed to learn to type numbers and other
LTR text backwards.

> But in the meantime it is very useful for working with Arabic text.

Again, users of Arabic scripts I talked to think otherwise: they want
bidi support, not just RTL display.

> > There is a third possibility in our case, using visual order within
> > Emacs and only storing the text in logical order.  That is possible in
> > a simple text editor (and I am sure there are some of those around).
> > But Emacs does a lot more, of course.  Every module in Emacs that
> > needs to look at the logical order would have to make the reordering
> > anyway.  And as Emacs is about text processing that would probably be
> 
> I don't see why.  Example?

The simplest example would be incremental search.  Suppose you type
"C-s ABCD", where upper-case letters denote RTL characters (e.g.,
Arabic letters).  If text is stored in the buffer in visual order,
Emacs will have to reorder ABCD into DCBA before passing it to the
text-search primitive.

Likewise, any other Emacs function that receives input from the user
or from external applications will need to do the reordering from
logical to visual order.  In other words, many places in Emacs will
need to be changed to handle visual-to-logical reordering.  This would
make the job of adding RTL support to Emacs unbearably hard.

In addition, while logical-to-visual reordering is a well-defined
operation, whereby for every logical-order string there's one and only
one visual-order string, the reverse is not true: for some
visual-order strings one can find more than one logical-order string
that, when reordered according to Unicode Bidirectional Algorithm,
will all give the original visual-order string.

> I just want RTL layout and Arabic shaping, both of which simply
> operate on a string of chars/glyphs.

But the Emacs developers want what the users of RTL languages want,
and that is bidi support, not just RTL display.  So any work done for
RTL support must be consistent with, and a part of, the full bidi
support, otherwise we, the Emacs developers, will object to including
it.

> The question of how best to represent text internally is an interesting
> one, but I haven't given it much thought.

We did give it much thought, and the conclusion was to use the logical
order.  That's why we must have bidi reordering in the display code.

> > A large part (maybe still a majority) of the people that write Arabic
> > and Hebrew on computers write in more than just one language.  This is
> > even if you discount numbers and trademarks.
> 
> Yes, I've heard this claimed many times, but I've never seen any
> evidence to back it up.  My personal experience is that it is simply not
> true.  In the Arab world, at least, *most* people do *not* operate in
> multiple languages (just like in the US), and from what I've personally
> seen they get along fine using Arabic only on a computer, just as most
> Americans get along fine using English only.

As I and others pointed out here, even Arabic-only text needs bidi
reordering because of digits and other weak and neutral characters.
The only way to avoid this reordering is to store characters within
Emacs buffers and strings in visual order, which we decided not to do,
for reasons explained above.

> In summary, there's more than one way to skin a cat, as the (American)
> saying goes.  Emacs (and other software) can be quite useful to RTL
> users without bidi support.  It's better to have bidi support,
> naturally, but the cost if bidi implementation need not stand in the way
> of providing useful stuff, and providing useful stuff by supporting
> non-bidi RTL and shaping need not inhibit implementation of bidi support.

This is Free Software developed by volunteers.  And all the volunteers
we have that know about and use RTL languages unanimously decided they
wanted RTL support with bidi reordering.  It is okay for you to
disagree, but the only method to steer Emacs development your way
would be to submit code changes to add simple RTL display without bidi
reordering to Emacs.  If you write such code, and it is clean and
doesn't get in the way of the future bidi reordering support, I
promise you I will review the code and recommend it for inclusion.

But as long as you leave this job to us, we will do what we think is
right, and that is RTL with bidi reordering support.