bug#56135: 29.0.50; python.el - forward-sexp regression over triple-quoted strings

[Kévin Le Gouguec]

João Távora <joaotavora@gmail.com> writes:

> On Wed, Jun 22, 2022 at 10:48 PM Kévin Le Gouguec <kevin.legouguec@gmail.com> 
> wrote:
>
>  Kévin Le Gouguec <kevin.legouguec@gmail.com> writes:
>
>  > João Távora <joaotavora@gmail.com> writes:
>  >
>  >> Why are you setting python-forward-sexp-function to nil?
>  >
>  > [wall of text follows]
>
>  (
>      Hadn't realized we had taken this off-list.  FWIW if you found all
>      my verbiage informative, don't hesitate to (tell me to) forward it
>      back on the bug tracker.
>  )
>
> Ups.  That was my mistake.  Wondering what to do now.  Should we re-send to 
> the 
> tracker.  Or let future generations wonder about the missing great wall of 
> text.

😁 I'll forward it in a separate message, in case someone finds other
points to bounce off of.

> Anyway, I did find your wall of text informative, particularly this point.
>
>> Wondering what the best way forward would be.  Maybe a third suggested
>> value for python-forward-sexp-function, which would handle triple-quotes
>> but not add those "implicit paired delimiters"?
>
> Yes, I think this makes sense.  Let's see what other think, but a patch
> that extracts only that bit from the current functionality and puts it in 
> a third value would be a good idea, I think.

Mm.  I've stepped through python-nav--forward-sexp with Edebug, and
there wasn't any obvious "triple-quote handler" to extract.  IIUC

* being on either end of a triple-quote block satisfies
  python-info-{beginning,end}-of-statement-p, so 'context' becomes
  'statement-{start,end},

* (then we dance a little gigue to compute next-sexp-{context,pos},
  which we do not end up using in this scenario,)

* eventually we call python-nav-{end,beginning}-of-statement.

So the "magic" happens in those functions IIUC.  E.g. in
python-nav-end-of-statement we either 

* skip the whole cond form because the initial (goto-char
  (line-end-position)) took us at EOL after a single-line string, or…

* go down the (setq string-start (python-syntax-context 'string)) path
  for multiline strings, or…

* … oh!  Mm, well, here's a bug maybe?  Consider:

"""string 1"""

def foo():
    """string 2

    (multiline).
    """
    print("""string 3""")
          ^

So *with python-forward-sexp-function set to its default value*
(python-nav-forward-sexp), with point on the ^ mark (at the beginning of
"""string 3"""),

* before commit 0646c68, C-M-f moved to the end of """string 3""",
* now it just moves over one pair of quotes.

(Feel like I should note that single-line triple-quoted strings have
their uses FWIW, since they let one write single and double quotes into
the string without the need to backslash-escape them)

I see that the new commentary in python-syntax-stringify addresses this,
I guess?

>           ;; FIXME: This makes sexp-movement a bit suboptimal since """a"""
>           ;; is now treated as 3 strings.

So AFAICT while the change meant to improve the situation with
electric-pair-mode (which I'm grateful for, since I'm a happy user of
that minor mode), it did make sexp-movement a bit more "janky" (meaning
"hard to predict and rely on", no offense meant), *regardless* of
whether users (un)set (python-)forward-sexp-function 😕