bug#8627: 24.0.50: cursor property behaves irregularly in before-strings

[Alp Aker]

The documentation for the cursor property (info node "(elisp) Special 
Properties") states that:

"Normally, the cursor is displayed at the end of any overlay and text 
property strings present at the current buffer position.  You can place 
the cursor on any desired character of these strings by giving that 
character a non-`nil' `cursor' text property."

And then follows the description of using integer values for the cursor 
property to make it applicable to a range of buffer positions other than 
the range between the overlay's start and end.

However, when used with overlay before-strings (or after-strings), the 
cursor property appears to behave in ways that aren't consonant with the 
docs and that don't follow a consistent pattern.

Here's a recipe for producing the variety of observed behaviors.  (I use 
before-strings, but using after-strings gives similar results.)

(1) Insert some boilerplate text to interact with.

  (save-excursion
    (goto-char (point-min))
    (insert "Lorem ipsum dolor sit amet.\n"))

Make an overlay and give it a before-string with a non-nil cursor property 
on one character:

  (setq olay (make-overlay 5 6)
        str1 (concat "XX"
                     (propertize "Y" 'cursor t)
                     "ZZ"))
  (overlay-put olay 'before-string str1)

If one now does:

  (goto-char (overlay-start olay))

the cursor property is ignored; the cursor appears after the 
before-string.  However, if the before-string appears before a newline, 
the cursor property is respected:

  (save-excursion
    (goto-char (point-min))
    (search-forward "\n")
    (move-overlay olay (1- (point)) (point)))
  (goto-char (overlay-start olay))

The cursor now appears at the propertized character in the before-string.

(2)  Now give the overlay a before-string one of whose characters has a 
numeric cursor property:

  (setq str2 (concat "XX"
                     (propertize "Y" 'cursor 1)
                     "ZZ"))
  (overlay-put olay 'before-string str2)

In this case, the property is respected regardless of location.  With the 
overlay still at the newline:

  (goto-char (overlay-start olay))

and in the middle of a row:

  (move-overlay olay 5 6)
  (goto-char (overlay-start olay))

the cursor appears at the propertized character in the before-string.

(3) But if the before-string contains a newline, the cursor property 
appears to be ignored regardless of location and regardless of whether the 
value of the cursor property is numeric or merely non-nil.  With the 
overlay still in the middle of the line:

  (setq str3 (concat str1 "\nWWW")
        str4 (concat str2 "\nWWW"))
  (overlay-put olay 'before-string str3)
  (goto-char (overlay-start olay))
  (overlay-put olay 'before-string str4)
  (goto-char (overlay-start olay))

the cursor appears after the before string, for both types of property 
value.  Now moving the overlay back to the newline:

  (save-excursion
    (goto-char (point-min))
    (search-forward "\n")
    (move-overlay olay (1- (point)) (point)))
  (goto-char (overlay-start olay))
  (overlay-put olay 'before-string str3)
  (goto-char (overlay-start olay))

the cursor property is again ignored, in both cases.

I started to read through xdisp.c to make some sense of this, but I'm too 
new to Emacs's internals to be able to follow much of the display routine. 
I did observe, though, that cursor_row_p only checks display properties, 
and doesn't appear to check before-strings and after-strings.

In any case, it would appear that either the treatment of before-strings 
and after-strings should be changed so that the cursor property works with 
them as it does with strings that come from display properties, or the 
documentation should be amended to make clear that that it can't be 
expected to do so.  (In the latter case, it would still seem desirable 
that cursor properties in before- and after-strings behave systematically, 
which isn't currently the case.)