bug#20847: [display engine] 25.0.50; company-mode popup makes point jump to an entirely different location

[Eli Zaretskii]

> Cc: 20847@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 21 Jun 2015 16:30:25 +0300
> 
> >   . There's no 'cursor' property on the overlay string that Company
> >     creates for its "tooltip" of completions.  Or at least I couldn't
> >     find that property: I tried both "M-x describe-text-properties" and
> >     looking at the overlay string in GDB -- I see no 'cursor' property,
> >     certainly not on the newline that starts the string.  If indeed
> >     there is such a property there, please tell how to see it.
> 
> I'm sorry, my memory of that code was unclear. `cursor' is currently 
> used only under certain rare condition, but see the end of 
> `company--replacement-string', where it's applied.
> 
> Even if remove the "when" condition to put it on all the time, the 
> behavior doesn't change.

If the 'cursor' property is put on the newline, then it indeed won't
change.  The support for 'cursor' assumes that the character on which
the property was put produces some glyph on the screen, because the
code which implements that works on screen lines (a.k.a. "glyph rows")
that were already laid out for display.  But the newline doesn't yield
any glyphs in this situation, so the fact that there was a 'cursor'
property there is left unknown to that code.

This cannot be fixed without a complete redesign of how 'cursor' is
supported.

> By the way, since we're now discussing the changes to Company code, 
> let's use the master version from 
> git@github.com:company-mode/company-mode.git. Thus, the scenario changes 
> from installing it from ELPA, to cloning the repo, adding the dir to 
> `load-path' and `(require 'company)'.

Ugh...  Why doesn't ELPA have the latest code?

> >   . In any case, you cannot put the 'cursor' property on a newline and
> >     hope it to work: the newline doesn't leave any glyphs on display,
> >     certainly not when the cursor is displayed on the fringe.  So the
> >     display engine doesn't know you've put the 'cursor' property there.
> 
> Which character should I put this property on in this situation, then?

The one before, I think.  Like I explained later:

> > E.g., perhaps
> > begin the overlay string a few characters earlier, so that it replaces
> > part of buffer text in "hel", and have the 'cursor' property on that
> > part of the string.

> The other odd thing is that the cursor is displayed in the margin 
> opposite the last completion (there are 2 or 3 of them), but the overlay 
> is actually of fixed size (10). So the cursor margin position doesn't 
> correspond to the value of point, nor either of the overlay bounds.

I didn't investigate why this happens.  Is it important?  Suppose I
"fixed" it by having the cursor on the same place in the second
paragraph where it ends up later -- will this be better in some sense?
If so, I will look into it.  (It's probably some redisplay
optimization that needs to be disabled in that case.)

> >   . The overlay string generated by Company in this case is
> >     problematic: it puts a newline at the end of each screen line, and
> >     that removes the last character of each screen line from display.
> >     I don't understand why you need to insert newlines when the
> >     original text lines were one long continued line.  This is a bug in
> >     Company.
> 
> It's a workaround I felt forced to take for the bug#18285. Since you've 
> vetoed my suggestion to prioritize `invisible' over `display', the 
> natural solution is to generally start the overlay a bit earlier.

Sorry, I'm not following, or maybe I don't understand the underlying
issues.  (I've re-read that bug, but I don't think it's relevant to
what I'm asking here.)  My question is about the newlines that get
inserted into the overlay string.  The original buffer text is one
long line, without any newlines, which occupies several screen lines.
Why cannot the overlay string be simply a copy of that text, a single
long string without any newlines?  The display engine will display
them with the continuation indicators on the fringes, exactly like it
does for buffer text, and the user will not "lose" those characters at
the end of each line.

> It also simplifies some logic in the code, because sometimes we have to 
> do this anyway (when the popup is displayed below the last line, and it 
> has no trailing newline).

You have "to do" what sometimes?

> > (Any
> > idea why Company's post-command-hook calls sit-for, thus forcing
> > redisplay?)
> 
> The current Company master has no `sit-for' calls, but both problems are 
> still there.

My last message exonerates these calls, they just got in the way when
I debugged the thing, and frankly surprised me, because having a call
to redisplay inside a post-command-hook that itself causes redisplay
by changing overlays sounds excessive.