Re: [emacs-bidi] Suboptimal display-reordering in minibuffer

[Eli Zaretskii]

> Date: Thu, 01 Jul 2010 15:37:35 +0900
> From: "Martin J. Dürst" <address@hidden>
> CC: address@hidden, address@hidden
> 
> Hello Eli,
> 
> On 2010/07/01 12:14, Eli Zaretskii wrote:
> 
> > You are suggesting to insert bidirectional format characters into the
> > buffer text in order to affect the display.  That's a no-no, IMO:
> 
> I agree that that's a no-no, for the reasons you give below. But that's 
> not what I was suggesting or thinking about.

Sorry.

>  What I was suggesting 
> (actually, the idea is originally from Kenichi Handa and/or Naoto 
> Takahashi) is that these bidirectional formatting characters go into the 
> text only 'virtually', e.g. in the before-string or after-string 
> properties of an overlay (see 
> http://www.gnu.org/software/emacs/elisp/html_node/Overlay-Properties.html#Overlay-Properties).
>  
> In that way, In my understanding, they are not part of the text buffer, 
> and will not be saved when saving the file.

Got it.

> Of course, if the characters in the overlay properties before-string and 
> after-string are not currently taken into account when running the bidi 
> algorithm, then that approach may not work very easily.

You are right: they aren't taken into account.  I have yet to code
support for reordering text in display strings.  To add this feature,
I will need to solve quite a few problems.  Until I do, I won't know
whether what you suggest is even doable with a reasonable effort.

I also think that, even if doable, this is a somewhat hackish
solution.  I think having a special text property that covers the text
that needs to be reordered is a cleaner solution.

> In any way, I think it's better to use the concepts already available in 
> the Unicode Bidi algorithm (override, embedding, marks) for improving 
> the display of XML, HTML, and other structured data and program source, 
> rather than to invent completely new concepts. Whether these concepts 
> then get transferred to the bidi algorithm via the (faked) insertion of 
> characters or via some other way (one could imagine to have properties 
> such as LRO/RLO/LRE/RLE on overlays,...) may be a secondary issue.

I think the upcoming Unicode 6.0 is already headed in that direction.
See http://www.unicode.org/reports/tr9/proposed.html#HL1.  The text
above this explicitly says that these provisions are for XML, HTML,
and other structured text.

So I think we will be fine doing it in Emacs.

> 1) it is easier for "application-level" emacs-lisp programmers who work 
> on updating modes to improve bidi display.
> 2) it is easier for the core implementer(s), i.e. you, because they have 
> to work with only one algorithm.

I don't intend to change the bidi reordering engine in any significant
way, to support these features.  All that's needed is a possibility to
tell it "restrict yourself to region between buffer positions P1 and
P2".  Actually, it just descended on me that I can easily do that with
`narrow-to-region', since the reordering engine already honors that,
it never goes out of the accessible portion of text.