Re: xterm-mouse-mode gives incorrect posn-object-x-y with display space

[Eli Zaretskii]

> From: JD Smith <jdtsmith@gmail.com>
> Date: Thu, 24 Nov 2022 16:42:29 -0500
> Cc: emacs-devel@gnu.org
> 
> Thanks.  The documentation on posn-object-* provides no guidance that a 
> “string object” would not include a
> stretched string with specified space.  So it was not at all surprising to me 
> that it worked "as expected" in the
> GUI.  In further support of this impression (leaving aside specified space), 
> posn-object-x-y works perfectly
> well on normal width characters in the buffer, providing pixel level 
> information on where *within the
> character* you clicked.  In fact I cannot think what "pixel-based x and y 
> coordinates relative to the upper left
> corner” would mean other than this for a "string object".  Surely that is not 
> an accident of the
> implementation!

posn-object-x-y for a display/overlay string measure from the nearest glyph
of the string character, not from the beginning of the string.

> > Yes, but on TTY frames the DX/DY _within_ the character glyph are always
> > zero, because each character is considered to be exactly one "pixel".  And
> > if I tell you that stretches of whitespace produced by 'space' display specs
> > are implemented on TTY as sequences of SPC characters (of course, what
> > else?), then I think you will understand why the DX/DY values, defined in
> > the ELisp manual as "the coordinates relative to the top left corner of the
> > character glyph clicked on", on a TTY are always zero: wherever the click
> > is, it is always on a glyph of some SPC character from those that represent
> > the stretch of whitespace.
> 
> Thanks for these details.  This I had indeed understood; with char units, you 
> can’t and won't get any pixel offsets within a single char glyph.  Since for 
> the TTY, Emacs implements a specified space under the hood as a “series of 
> spaces”, the natural thing for consistency with the GUI would be for 
> posn-object-x-y to report click position offsets (in integer character units, 
> naturally) within that stretched space.  It sounds like this may be 
> challenging to implement internally, but it would be the most consistent.  It 
> must be possible, since Emacs 1) knows it drew a specified space as a series 
> of space chars in the TTY, and 2) knows (or could know) where the click 
> occurred within that group of space chars it drew.

The problem is that the "knowledge" about stretch implementation is on a
much lower level than the one on which posn-object-x-y operates.  On the
level of posn-object-x-y there's no stretch, just the position and the
metrics of the glyphs it produced.

> To summarize, the fact that in the TTY a stretched space is implemented as a 
> collection of space chars, and on GUI, some other type of pixel-precise 
> graphical element is IMO just an implementation detail.  What if TTY’s of the 
> future allow drawing small graphical elements at pixel precision? How is the 
> user of posn-object-* to know which of the two different and incompatible 
> result flavors they will get?

If that ever happens, "Someone" will have a lot of work on their hands.

> This is how I implement mlscroll, a mode line scroll bar we’ve discussed here 
> before.  The basic design is 3 specified spaces in a row , usually 
> right-aligned in the mode line. Together these 3 (varying width) spaces 
> indicate the number of lines: i) above ii) showing, and iii) after the 
> window.  Just like a “real” scrollbar, the user can interact with it by 
> clicking/dragging/etc.  When clicking on the bar, the precise relative click 
> position determines what lines are scrolled into view (again, just like a 
> real scrollbar).  Surprisingly it looks pretty good on TTY without any 
> special casing, though obviously with char- instead of pixel-granularity.  
> But this happy consistency breaks down with xterm-mouse-mode, since clicks on 
> the modeline scrollbar are not reportable as positions w.r.t. the clicked 
> element, i.e. since posn-object-x-y always reports 0 on TTY.  

I don't understand what you mean by "surprisingly it looks pretty good on
TTY without any special casing", if mouse clicks on TTY frames give you this
problem.  Are you saying that you only see this with xterm-mouse-mode, but
not with some other mouse which works with Emacs TTY frames?

> >  (let* ((event-posn (posn-point (event-start ev)))
> >         (click-x-y (posn-x-y (event-start ev)))
> >         (obj-x-y (posn-x-y (posn-at-point event-posn))))
> >    (cons (- (car click-x-y) (car obj-x-y))
> >          (- (cdr click-x-y) (cdr obj-x-y))))
> > 
> > IOW, don't trust DX/DY, but calculate the offsets "by hand".
> 
> 
> Thanks for your good suggestion; this would indeed work as desired in the 
> buffer, since posn-at-point does seem to “know” it’s in a specified space in 
> both GUI and TTY (which makes me think posn-object-x-y should also be able to 
> know this).
> 
> Unfortunately posn-point returns nil for posn-area = mode-line, so this does 
> not solve the problem.  Is there any other way you know to determine the 
> starting char position of the _group of spaces_ that TTY Emacs translates a 
> specified space into, in the mode line?  Calculating the char-start of the 
> scrollbar myself will be challenging given dynamic modeline elements and the 
> possibility for the user to place the scrollbar anywhere they want (i.e. not 
> just right-aligned).

Why challenging?  On TTY frames, align-to always aligns to some character
position, and the mouse clicks are reported in terms of character positions
as well.  Why do you even need the fine details of the POSITION part of the
mouse-click events on TTY frames?