bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n)

[Eli Zaretskii]

> Date: Sat, 5 Jan 2019 22:53:14 +0200
> From: Khaled Hosny <dr.khaled.hosny@gmail.com>
> Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org
> 
> > Done.  Please test.  I made sure it compiles, but I couldn't actually
> > test the results, as I don't have access to a GNU/Linux system with
> > GUI display.  So it could be that I misunderstood the Harfbuzz APIs,
> > as I was essentially flying blind, guided only by the Harfbuzz docs.
> 
> It seems to work, but still not quite right. You seem to be passing the
> paragraph direction, but what HarfBuzz needs is resolved direction of
> the text (i.e. the bidi embedding level of the run). In other words, if
> Emacs is going to draw this text from right to left, then HarfBuzz must
> shape it in right to left direction. Both should use the same direction
> all the time and HarfBuzz direction guessing should never be used (i.e.
> always pass to it an explicit direction).

In response to that, I wrote:

   It isn't the paragraph direction; at least it wasn't supposed to be
   that.  The code is (or was before your changes):

         if (charpos < endpos)
           {
             if (pdir == L2R)
               direction = QL2R;
             else if (pdir == R2L)
               direction = QR2L;
             [...]
             cmp_it->reversed_p = 0;
           }
         else
           {
             [...]
             cmp_it->reversed_p = 1;
             [...]
             if (pdir == L2R)
               direction = QR2L;
             else if (pdir == R2L)
               direction = QL2R;
             [...]
           }

   So, as you see, when the paragraph direction is L2R, normal text gets
   L2R direction, while test reversed for display gets R2L, and the other
   way around when the paragraph direction is R2L.  Which AFAIU is what
   HarfBuzz needs, but maybe I'm missing something.

   Did you actually see incorrect display with the code I wrote?  If so,
   could you please show the recipes for reproducing that, preferably
   with screenshots of correct and incorrect display?  I'd like to look
   into that, to understand what I missed.

   > HarfBuzz direction guessing should never be used (i.e.  always pass
   > to it an explicit direction).

   This is in general impossible (or at least very hard), since the
   shaper is sometimes called from Lisp without any display context.  See
   the Lisp callers of the function font-shape-gstring.  One use case is
   when we want to display the composition information for a grapheme
   cluster to the user, see descr-text.el (used by the "C-u C-x ="
   command).  In these cases, the UBA is not invoked, and so we don't
   have the direction information.

   I could provide the direction information in this case by using the
   directionality of the base character of the grapheme cluster, but I
   figured out that HarfBuzz already does this as part of its guessing.
   Doesn't it?

   > I pushed a couple of commits that does this based on my limited
   > understanding of Emacs code, please check.

   Thanks.  Do you see any difference in the results?  If so, can you
   please show the text you used and the results of shaping it with both
   versions.  AFAIU, your code should produce exactly the same results,
   unless I'm missing something.  (I didn't want to use the
   resolved_level attribute because it is ephemeral, and might not
   provide the correct value where we are using it.)

   Btw, did you test both paragraph directions (controlled by the
   bidi-paragraph-direction variable), and also text inside directional
   override which changes its natural direction?

Could you please respond and answer the few questions I asked?  I'd
like us to continue working on the branch.

TIA