Re: [emacs-bidi] Callbacks?

[Eli Zaretskii]

> Date: Thu, 01 Dec 2005 20:21:07 +0900
> From: Martin Duerst <address@hidden>
> Cc: address@hidden, address@hidden
> 
>  >That's actually a bad idea, IMHO: the text should be saved so that any
>  >other bidi-enabled editor will display it the same.
> 
> For plain running text, this is definitely true. I'm not sure
> this applies also for structured stuff such as XML.

This applies in general, IMO.  We could make exceptions in specific
cases, but only if there's no other reasonable way to solve the
problem.  Interoperability with other software is a very important
goal, otherwise we wouldn't be considering strict compliance with
UAX#9 (which clearly made wrong choices at some strategic points).

>  >That is why
>  >fiddling with Unicode character properties is something I feel we
>  >should not do: those properties are the only common denominator of all
>  >bidi editors.
> 
> I definitely understand your feeling. But I just want to mention
> that the Unicode bidi algorithm explicitly allows things like this.
> If you look e.g. at:
> http://www.unicode.org/reports/tr9/#HL3 and
> http://www.unicode.org/reports/tr9/#HL5,
> you won't see changing of properties mentioned explicitly,
> but you'll be able to figure out that "Provide artificial
> context" does very much equivalent things.

I think these clauses of UAX#9 are compromises: some software vendors
need to produce correct display without supporting directional marks
and overrides, so they talked the Unicode consortium, into adding that
language.  IIRC, it wasn't there in the original specification.

> Let's take a very simple example, first logical:
>     <x y='ABC'>DEF ghi</x>
> Not applying anything but the bidi algorithm, this gets displayed as
>     <x y='FED<'CBA ghi</x>
> which I hope you can agree is a useless mess;

Yes.

> what we would like to see is something like
>     <x y='CBA'>FED ghi</x>

Yes.

> Now to get that, it would be okay to add a single [LRM],
> e.g. like so (logical):
>     <x y='ABC'[LRM]>DEF ghi</x>
> But with the [LRM] in this position, you will get an
> XML parsing error.

I didn't necessarily mean to put a literal LRM in the file, only in
the Emacs buffer.  In the file, we could use something else, like the
directional markup similar to HTML.

> In this specific case, there are
> two other possible positions for the [LRM] (logical):
>     <x y='ABC[LRM]'>DEF ghi</x>   and
>     <x y='ABC'>[LRM]DEF ghi</x>
> But now, we have made the [LRM] part of the attribute
> value or the element content. In both cases, we have
> changed the content of our *data* just to solve some
> display problems of the *markup*. It should be clear
> that this is completely out of question

I'm not sure why.  LRM and its ilk are just characters, albeit
normally invisible ones.  Why is it ``absolutely out of the question''
to have them as part of the strings in XML?

> Now about using HTML directives (the HTML dir attribute,
> and the <bdo> element): Again, these are for indicating
> directionality of the marked-up content. Trying to use
> them directly to fix some markup details as in the examples
> above doesn't seem appropriate.

Sorry, I don't see the difference between indicating directionality of
marked-up content and fixing markup details.  I think we _are_ trying
to solve the issue of markup content directionality, as the only
parts whose directionality needs to be specified are the data, the
strings that are going to be displayed/interpreted by the XML parser
and its clients.

> In any case, we are still left with two problems:
> Generic XML does not have these directives

Then generic XML is inappropriate for bidi text, and it should be
fixed (or a variant of XML developed that does support directional
directives).

> we don't have an editor yet that would react in
> the right way to such directives.

Once these directives are mandated by some standard, editors will add
support for them.  Yet another reason not to invent your own private
solutions, but talk to standards people to do that, IMHO.