Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan!

On Tue, Feb 03, 2009 at 09:21:02PM -0500, Stefan Monnier wrote:
> > Visit src/buffer.c, goto L313 which contains the closing brace of
> > DEFUN Fget_file_buffer, put point AFTER the "}".  Or put point on the
> > next line (L314).  In either case C-M-e leaves point at BOL315.  

> Oh, so we just disagree about what is the correct behavior.  As far as
> I'm concerned, if C-M-e gets you to BOL315, that means that the end of
> the function is BOL315, which implies that EOL313 is inside the
> function, so it's correct for C-M-e to just move to BOL315.

WADR, this is sophistry.  Correct behaviour is to go to the end of the
defun.  In C, this is the closing }.  It CAN'T be anywhere else.  Take a
straw poll of 100 C hackers and ask them.  It's the place where a
compiler would give an error message if you chop off the buffer before
it.

The doc string for `end-of-defun' is crystal clear here: "Move forward to
next end of defun".

> > If one were to define a variable end-of-defun-RAW-function, which is
> > defined to leave point just after the last bit of the function,

> That's what end-of-defun-function should do, indeed.

I would agree.  I detest sloppy definitions, but EOD-function has been
sloppily defined for years.  I think we should accept that EOD-functions
in existing modes DON'T all go to this ideal point, but themselves
correct for WS.

> > perhaps that would work, but it would break existing 3rd party code.

> Like which code?

Well, in my first page of 50 in a google search for
"end-of-defun-function", I came up with these, all of which set
end-of-defun-function:
cfengine.el
espresso.el ; a Javascript mode.
lua-mode.el
js2-sl-mode.el ; One of Lennart Borgman's files
php-mode.el
Slime mode ; probably slime-mode.el

There are certainly several more.  There are likely many more.  Some of
them will do WS jiggling at the end.  I don't think we should make these
end-of-defun-functions broken.

> >> Unless by "the problem" you're talking about the performance problem, in
> >> which case I understand that each time we call BOD (except for the first
> >> call), we know that we're "outside" of a defun (unless there's nesting,
> >> of course) but we don't tell that to BOD which may have to redo the work
> >> of figuring it out.

> > WILL have to do the work.

> No: in most cases, beginning-of-defun-raw just uses a simple regexp and
> doesn't try to determine its position in the syntax tree before doing
> its job.  This is pretty much how C-mode's beginning-of-defun worked in
> Emacs-21 and incidentally it worked (for the cases I care about) much
> better than the current code.

I don't know enough major modes to talk about most, for whatever value of
"most" you have in mind.  I hold it to be wrong to assume that any
BOD-function or EOD-function is implemented "simply", whether or not with
a regexp.

C Mode's beginning-of-defun in Emacs 21 was broken, but that's really a
different matter.  I don't claim that C Mode's C-M-e is optimal.  It is
however reasonably good and it's fast enough for nearly all C files in
Emacs 22.  Any place where it's not fast enough will find its C-M-e's
speed halved or quartered in Emacs 23, and C-u n C-M-e will be slowed
down to loss-of-temper speed.

> > Now, I wonder, who could that have been?  ;-)  Oh, how I'd love just
> > to get rid of these stupid K&R declarations, which NOBODY uses at all
> > nowadays, except on >20 year old code bases.  ;-)  Why don't we flush
> > them out of Emacs C source?  (No, you don't have to answer.)

> Feel free to remove support for those things in C-mode.

A user option c-recognize-k&r would be the way to go, I suppose.  It
would be trivially easy to implement and would improve the performance of
C Mode.  I was thinking more of purging the K&R declarations from the
Emacs source code.  Perhaps for Emacs 24?  This surely would be a forward
step now, even if it wouldn't have been in 1990.

> If that can solve the performance issues, I'm all for it.  After all,
> Emacs-21 didn't get it right and I haven't heard significant complains
> about it.  It's not like Emacs is always expected to understand every
> little bit of a language's syntax anyway.

Well, we could get philosophical about this.  I think that every little
glitch which "isn't quite right" detracts from quality, and if there are
enough such glitches, the user's impression will be of general low
quality.

> >> > It's changed from "move to next end of function" to "move to the
> >> > end of the function at whose beginning we now are",

> >> Right.  As you may notice, the second is a subset of the first (with
> >> a few caveats for nested functions, of course, but that shouldn't matter
> >> for C-mode), so if your implementation works for the first, it should
> >> work for the second as well.  It's called backward compatibility.

This change has caused grief and confusion already, and will cause more
if it gets out to users.  I don't think there is backward compatibility.
In the near future, some EOD-functions will only work from a BOD, some
will work from anywhere, and this will break any 3rd party code which
calls end-of-defun-function (assuming old semantics) for other purposes.

Let me repeat that:  THIS CHANGE WILL CAUSE EXISTING CODE TO BREAK WITH
FUTURE IMPLEMENTATIONS OF end-of-defun-function.

> > c-end-of-defun's functionality "needs" to stay the same to support
> > other Emacsen.  For Emacs-23's EOD-function, It would barely need to
> > be more than
> >      (search-forward-regexp "{")
> >      (backward-char)
> >      (foward-sexp)

> Yes, simplicity of implementation of EOD-function was indeed an
> important motivation in my code changes.

Is there any win with this?  Any hacker writing an EOD-function can get
this simplicity himself, if it's appropriate.

> >> > and its default value is `forward-sexp'.  `c-end-of-defun' was a
> >> > good fit for the variable as it formerly was, but is now severely
> >> > suboptimal.

> >> At most by a factor of 2.  I.e. if it's slow now, it sure wasn't
> >> zippy before.

> > By a factor a great deal more than 2, highly dependent on ARG.

> The only case I care about for now is when ARG=1.  As long as this case
> is slow, there's no point complaining that end-of-defun makes it too
> slow for ARG=106.

It isn't slow for ARG=1, except for some ill-conditioned files (perhaps
src/lisp.h).  In these cases, even a factor of 2 is worth the trouble.
I'm currently working on one of the big slowness problems.  Please
identify any problematic files you know of whose slowness isn't caused by
a paucity of braces.

Somebody other than me will care about ARG > 1.  She'll care a lot when
Emacs 23 hangs for 5, 10, 20, 50 seconds.  People who will use C-M-e to
move large distances over files, for example.  In Emacs-22, this is fast
- the speed is essentially independent of ARG.  Why should we slap them
in the face when we release Emacs-23?

> >> >> Not sure about restoring the previous semantics.  But I could
> >> >> agree to the additional ARG argument, which could even let it
> >> >> "take over" (so beginning-of-defun-raw is not called in that
> >> >> case).
> >> > :-)  Let's do it!
> >> I knew you'd like it.
> > :-)

> I'm still wondering whether we couldn't just turn end-of-defun into
> a simple variant of

>    (BOD-raw N)
>    (funcall EOD-function)

It won't work when point is outside a defun.  And a combination of (BOD)
and (EOD) isn't capable of reliably determining if point is outside a
defun.  This was the main point of my last post, which you didn't
address.  To use this approach, a new hook would be needed, say
`inside-defun-function'.  Then end-of-defun would become:

    (when (funcall inside-defun-function)
          (setq N (1+ N)))
    (BOD-raw (- N))
    (funcall EOD-function)

Well, something like that.  It would be surely be worse than simply
(funcall end-of-defun-function N).

This approach would probably be making hidden assumptions about nested
defuns, or other wierdly formulated defuns where BODs and EODs don't
necessarily alternate.  Also any 3rd party code which uses EOD-function
will be assuming that EOD-function works with point anywhere.  If we're
going to go this road, the (simplified current) EOD-function should get a
different name; perhaps `forward-defun-function'?  This would underline
its unlikeness to beginning-of-defun-function.

I honestly don't see the advantage to this.  Why don't we just assume
that if begininng-of-defun-function and/or end-of-defun-function have
been defined (with the previous semantics + &optional arg) they will do
their job properly?

> > OK.  I think I can see what you're saying, now.  Were you thinking of any
> > (non CC Mode) languages in particular?  Did this ever get discussed on
> > emacs-devel?  I might well have been asleep at the time.

> No, I don't remember discussing it much.  It did come out of
> a discussion where it became apparent that end-of-defun was broken
> (especially when called with negative args).

OK.  Maybe the elisp manual should emphasise the possibility of
negativeness.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).