Re: bidi-display-reordering is now non-nil by default

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Let's stick to the issue at hand: do you consider it a good idea to
 > remove or add these characters in an otherwise unmodified buffer?

Er, you're the one who keep bringing up random irrelevancies like
byte-level equality.

And your question makes no sense.  In your implementation, they will
be present as characters in the buffer, and they should neither be
removed nor added per the Unicode standard.  In a text-property-based
implementation, on input they will be converted to text properties on
the characters controlled, and automatically converted back on
output.  Once again, those characters will neither be removed nor
added in the buffer.

 > > I see no reason why a text-property-based implementation should be
 > > lossy.
 > 
 > Because the user could type directional controls, and there's no way
 > for Emacs to know at all levels which one is to be treated in which
 > way.

What's wrong with reparsing the buffer from the beginning, treating
each change of value of the direction property as insertion of the
appropriate direction mark?  If there are redundant marks, of course
they would have to be indicated in some way.  But if that doesn't
work, I don't see how having explicit mark characters in the buffer
can work either.

 > > I don't understand.  If `put-text-property' and friends don't get that
 > > right already, more than bidi is in trouble, I should think.  What's
 > > special about bidi?
 > 
 > What is special is the fact that bidi needs nested regions with
 > different values for the same property.  Normally, if you put a
 > property with a value on a portion of text that has another value for
 > that property, the new value replaces the old one.

Sure.  But this is Lisp.  There's nothing that says that you are
limited to something as simple as 'ltr vs. 'rtl as the property value.
You could have a rather complex property, eg, containing the level of
the embedding as well as the resolved direction.  Or you could simply
replace the directional marks with a string on the preceding non-mark
character containing the mark characters that were present in the
source.  (Of course you'd need some hack for marks at the beginning of
buffer.)

It might not be worth it.  But then again...