bug#56135: [Kévin Le Gouguec] Re: bug#56135: 29.0.50; python.el - forward-sexp regression over triple-quoted strings

[Kévin Le Gouguec]

(Off-list message that João already replied to in
<CALDnm53bj4rWzkP50L7TdwF3CRpy4rST3oy23bujWgC9YCJnAA@mail.gmail.com>;
re-sent for archival)

-------------------- Start of forwarded message --------------------
From: Kévin Le Gouguec <kevin.legouguec@gmail.com>
To: João Távora <joaotavora@gmail.com>
Cc: Stefan Monnier <monnier@iro.umontreal.ca>
Subject: Re: bug#56135: 29.0.50; python.el - forward-sexp regression over
 triple-quoted strings
Date: Wed, 22 Jun 2022 22:39:42 +0200

João Távora <joaotavora@gmail.com> writes:

> On a very superficial analysis, I would say you're _not_ experiencing a 
> bug, because setting the f-sexp-f to nil buffer-locally would mean
> you don't want any python-specific behaviour for triple quotes.  
>
> It might be a regression for a previous state of the working tree, but then
> again that state seems to have been buggy itself, undoubtedly, at least as 
> described by myself in 0646c6817139.  
>
> Why are you setting python-forward-sexp-function to nil?

The motivation behind that user option (or the commentary suggesting to
set forward-sexp-function to nil before the user option existed) is to
change how python-mode's *ward-sexp commands behaves wrt blocks.
Consider this function:

    def foo():
        return bar_baz(quux, corge)

By default, when point is…

* at the start of "def": C-M-f moves to the end of the function (at the
  end of "bar_baz(quux, corge)"),

* at the end of "bar_baz(quux, corge)": C-M-b moves to the beginning of
  the function (at the start of "def").

IOW by default python-mode uses sexp commands to move across whole
blocks, even when point is on something that might be considered a sexp
in other major modes (identifiers, or paired delimiters); I imagine that
since Python blocks are based on whitespace rather than explicit
delimiters, python-mode tries to be helpful by adding "implicit paired
delimiters":

    (a)[b]def foo():(b)
           (c)return bar_baz(quux, corge)[c](a)

(Read "(x) jumps to the corresponding (x)"; using [y] with square
brackets to designate "destination points" for C-M-f/C-M-b that cannot
be used as "source" points for C-M-b/C-M-f, since they are shadowed by
(x) "source" points in the same spot)

Setting (python-)forward-sexp-function to nil was the recommended way to
take those implicit paired delimiters out of the equation[1].


So that was the situation before this commit, AFAICT: triple-quote
docstrings did not enter the picture and even with forward-sexp-function
set to nil, C-M-[bf] did TRT (skipping over them).

I can readily believe that things working the way I liked was a happy
accident and, on balance, your change fixed more than it broke.

Wondering what the best way forward would be.  Maybe a third suggested
value for python-forward-sexp-function, which would handle triple-quotes
but not add those "implicit paired delimiters"?

FWIW my impression (from some light Googling) is that setting this
option to nil (or the forward-sexp-function variable buffer-locally) is
something that a fair number of python-mode users have been doing for
years, without worrying about how that would impact docstring
navigation.


[1] Irrelevant motivation tucked under a footnote:

    Getting rid of the "implicit paired delimiters" suits me well, since
    it's easier for me to reason about:

    * I frequently overshoot my movement and need to hit e.g. a couple
      of back- commands after a string of forward- commands;
      python-mode's those invisible pair delimiters make
      {forward,backward}-sexp no longer cancel each other, which is
      disorienting;

    * if point is on a word or a paired delimiter, I expect *ward-sexp
      to move over that, instead of leaping over a whole block.

    E.g. when point is on the closing parenthesis of "bar_baz(quux,
    corge"): if I want to move to the start of "bar_baz",

    * with "traditional" sexps, it's just two C-M-b away,

    * with python-mode's default sexps, it's… I dunno; M-b C-M-u C-M-b?
      M-m M-f C-f?  It's frustrating because in any other major mode, I
      don't need to _stop and think_ about how to solve that puzzle;
      those invisible pairs force me to hold my fingers, squint and
      guess their position, then plan my route.

    I'm sure python-mode's default block movement is useful to some
    folks; IME I do fine with defun-movement (C-M-[ae]) or python-mode's
    other block commands (M-[ae]).  I get more value out of C-M-[bf] by
    keeping them limited to identifiers and pairs.
-------------------- End of forwarded message --------------------