Re: [emacs-bidi] Bidi Emacs: R2L paragraphs and the direction of glyph_row

[Eli Zaretskii]

> Date: Fri, 27 Nov 2009 20:11:47 +0900
> From: "Martin J. Dürst" <address@hidden>
> CC: Gregg Reynolds <address@hidden>, address@hidden, address@hidden
> 
> On 2009/11/27 1:10, Eli Zaretskii wrote:
> >> Date: Thu, 26 Nov 2009 08:58:39 -0600
> >> From: Gregg Reynolds<address@hidden>
> 
> >> It's also a major pain to edit XML in a bidi editor.  And I do mean
> >> major.
> >
> > Again, an explicit list of main problems would be useful (although I
> > deliberately decided not to handle bidirectional editing for meta
> > documents yet, so this specific class of problems is still a long way
> > from being dealt with).
> 
> This is a well-known problem. For some more background and some attempt 
> at a solution, please see:
> 
> http://www.sw.it.aoyama.ac.jp/2005/pub/IUC28-bidi/ and
> http://www.sw.it.aoyama.ac.jp/2008/pub/IUC32-bidi/

Thanks.

As I pointed out elsewhere, this class of problems is currently beyond
the scope of what I'm trying to do in Emacs.  That is because the
display engine basically examines characters one by one, so correct
display of XML/HTML using just UAX#9 algorithm is impossible, as you
and others have found out.

I think that support for bidi editing of documents that use markup
languages should be based on the features that exist as part of the
markup: &lrm;/&rlm;, the dir= directive etc.  Using these, a Lisp
application should place overlays or maybe text properties on the
portions of text that need to be reordered.  For example, one could
place an overlay on a portion of the buffer with `display' property
whose value is the reordered text of that portion (it will have to be
enclosed in LRO..PDF embedding to avoid the automatic reordering when
the display string is delivered to the glass).  Or we could add a new
overlay property that would just tell the display engine "reorder this
part".  Then the display engine will DTRT guided by these overlays and
text properties.

However, please note that the above is not something I thought about
long enough to be convinced that it is the right solution.  The only
thing I convinced myself in at this point is that bidi support for
markup languages should not be on the basic display engine level,
which is what I'm currently hacking.  Rather, it should use the basic
reordering capabilities of the display engine in some way, but do some
additional work on higher levels to implement the equivalent of the
UAX#9 ``higher protocols''.

> In a discussion with Ken'ichi Handa and Naoto Takahashi in 2005, they 
> pointed out that using bidi marks in overlays should make it possible in 
> bidi emacs to address this problem. Unfortunately, nobody here is 
> familiar enough with emacs lisp to implement this, but I'd be very glad 
> to help somebody with some emacs lisp skills to work on such a project.

Thanks for the offer.  I don't know when I will get to those parts.
For now, text properties and overlays are not yet supported in the
modified display engine -- that is the very next problem on my agenda.
When these parts are done, hopefully others could come on board and
implement markup support on top of that.