Re: [emacs-bidi] state of BIDI within stable GNU emacs (UTF8) nikud

[Eli Zaretskii]

> Date: Tue, 29 Nov 2005 18:13:58 -0600
> From: Gregg Reynolds <address@hidden>
> CC: address@hidden
> 
> I suppose I also have to sign some kind of papers.  Probably not a
> problem with my employer, but the timing is a bit delicate at the
> moment.

We only need papers for changes longer than 15 lines.

If you cannot sign papers, then just explaining (in English) where is
the buggy code and how, in principle, it should be changed, will do;
someone else who already signed papers (such as myself) can then code
the required change.

> Hmm.  I wonder if we couldn't come up with stand-alone test stuff that 
> would establish the capability of the code w/r/t emacs processing first, 
> and then do in-context emacs testing.  Speculative question.

I thought about this at the time, and decided it would require a
significant effort that wasn't worth it.  One big problem is that I
don't think I know enough about the redisplay code to be able to
specify all the different circumstances under which the low-level
iterator code (into which my bidi code is plugged) is called.  It's
better to let Emacs speak for itself ;-)

> Just for the sake of clarity and hygiene.  In general, if I'm reading a 
> chunk of code, I'd rather not be distracted by code that really should 
> be in a distinct test harness.  Plus, would your bidi code not be of use 
> to anybody outside of emacs?  Looks to me like it might be a useful 
> alternative model to other bidi implemenations.  If so, would it make 
> sense to make it available as a stand-alone library?

I'm not sure what other applications would need a sequential
implementation.  For starters, if there are any, there should have
been such an implementation out there, but there isn't.  The Emacs
display engine is quite unique in this regard (and for good reasons).

> I've never tried to test an implementation against UAX9, myself.  But 
> Unicode bidi stuff is pretty complicated - after all, here we are 10+ 
> years into it, and *still* it has not stabilized (revisions are now 
> pending).  After all, we're talking about 61 levels of embedding, 
> multiple directionality classes, etc. etc.  I don't have the slightest 
> idea how one could verify all that crapola.

I built many small test cases which use every feature.  To that, I
added a test case for every bug I've found during debugging.  As for
embedding, it's sufficient to test several levels, no need to test all
61 of them.

> You mentioned that the implementation "passes with flying colors".  Let 
> me play the devil's advocate:  prove it.

Well, this is Free Software; I don't have to prove anything, since no
one is paying me to produce bug-free code ;-)

> As for testing the implementation in emacs, that's gonna be pretty 
> tough.  There's a lot of complexity in there, there.  E.g., how will it 
> work with elisp programs?  What if somebody mucks about with the various 
> tables that touch on characers?  Wants to use an obscure encoding? 
> Etc., etc.  I would think that test/log frameworks would be of immense 
> help in that sort of testing.

Logging is going to be our smallest problem for that.  We will need
first to write code that finds what's useful to log in each case.  I
say let's start with that; we can add sophisticated logging later, if
needed.  Until now, debugging of the display engine--a very complex
piece of code--didn't need it.

> Hmmm.  Seems to me we should be able to automate even display logic 
> testing.  As I hinted above, it looks to me like test/log frameworks 
> should actually reduce complexity.

Yes, the lack of a test suite in Emacs is a significant drawback; it
was discussed on the developers' list a year or so ago.  But hey, this
is emacs-bidi mailing list; we don't need to solve _all_ the problems
in Emacs, only this tiny one ;-)