Re: emacs rendering comparisson between emacs23 and emacs26.3

[Dmitry Gutov]

Hi Alan,

On 12.04.2020 18:34, Alan Mackenzie wrote:
> > > But this would merely transfer the start up time to the
> > > time taken in early scrolls forward.
>
> > Not really. The start up scans the whole buffer, doesn't it? The early
> > scrolls forward would still scan only a fraction of it.
>
> I'm thinking more about "scrolling" to the function in the file that one
> wants to work on or look at.  On average, this will be a little more
> than half way through the file (there is often a large comment block at
> BOB).  So you'd only be saving about half of CC Mode's start-up scan.

Yes, on average it's only 2x benefit, but then again, if the buffer 
opens at its beginning first, the extra initialization will be spread 
across user commands. Which should have some additional positive effect 
on apparent responsiveness.

> > Inserting characters can alter the syntax state of the whole buffer. At
> > least that's true for some of them. Full buffer scan sounds inevitable
> > in those cases.
>
> Full buffer scans are very unusual.  Inserting a " where every
> subsequent line ended with a backslash might do that.  Inserting a C++ <
> wouldn't - its effect is limited to up to the next brace or semicolon.
>
> Inserting a C++ raw string opener does typically necessitate a full
> scan (a search for a matching closer), but that would also be the case
> using syntax-propertize.

Not really. It would just mark the opener as a string opener (maybe with 
some extra text property), and that's that. Then font-lock would fontify 
the following text as string contents (until the end of the window or a 
little bit farther). Then you type the closer, it only has to scan a 
little far back (it'll call syntax-ppss to find the string opener), the 
closer is propertized as appropriate, and that's that. No full buffer 
scans at any step.

I recall that fontifying the rest of the buffer as text after a simple 
string opener could be a sore topic for you, but raw strings should be 
rare enough (aren't they?), or if they are not, fontification logic 
could opt to do something different, while syntax-table properties will 
be applied the "correct" way.

> So, we would merely be moving functions from
> c-get-state-before-change-functions and c-before-font-lock-functions
> (effectively lists of before-/after-change functions) to
> s-p-extend-region-f, together with adaptation. Would you agree that
> such a change to CC Mode would be largely pointless if some of these
> functions had to remain on c-get-state-b-c-f and c-before-f-l-f?

Yes, I think before-change-functions should become empty. Or much emptier.

> But the way s-p-extend-region-f functions are called is to keep calling
> them repeatedly until they've all said "no change" together.  This would
> dramatically slow down CC Mode, where currently these functions are each
> called exactly once.

Here's a simple solution: create one function special for CC Mode that 
would do that, and add it to s-p-extend-region-f.

But there might be even better way to do that. I'm not the best person 
to discuss that with.

> Also, the syntax-propertize mechanism is weaker than CC Mode's: When it
> is run, there is no way of knowing whether it's being called as a change
> function, and if it is, OLD-LEN is discarded.  How can it have access to
> variables set in before-change-functions?  (An example of such is
> c-raw-string-end-delim-disrupted.  In before change, it is set when the
> existing raw string end delimiter is about to cease to be.  In after
> change, the fact of this flag being nil means we don't need to search
> for an alternative closing delimiter, etc.  This change can obviously
> not be detected in an after-change function.)

As we seem to agree, before-change-functions should not be needed. 
Neither should be the code that tracks the details of the edits that the 
user makes. That alone can simplify some things.

> > I'm pretty sure I have thought of that example because it's an instance
> > of a syntax problem that's easy enough to solve within
> > syntax-propertize-function framework.
>
> Having actually gone through all the issues and implemented raw strings,
> I can't agree with you there.  There are all sorts of subtleties which
> necessitate intimate cooperation between the before-change-functions and
> after-change-functions.  Such cooperation seems to be excluded by the
> syntax-propertize mechanism.

It encourages a different approach. Again: there are examples of raw 
strings support in other major modes.

> > > Consider(2) a C++ template: excusing my C++ syntax knowledge, type in
>
> > >      template class foo < bar, baz >= bar>
>
> > > , perhaps typing in the odd newline inside the template (a common
> > > occurrence), or nesting further templates inside it (also a common
> > > occurrence).  Note how the parenthesis text properties are added and
> > > removed as you type.  All these modification are necessary, and they are
> > > largely _before_ the point of insertion, not after it.
>
> > The current implementation of applying these properties can probably be
> > transferred into a syntax-propertize-function with only modest changes.
>
> Maybe, but with a slowdown.  More of these properties will get erased
> than needed (with nested template forms), and they will all need to get
> put back again.

We won't really know until we can measure the result.

> > > > Some scenarios can become slower, that's for sure. But the more common
> > > > ones can get faster. We won't know until we try.
>
> Other than starting up a buffer, we still haven't identified any
> specific scenarios where speed up might happen.

When before-changes-functions only contains (t syntax-ppss-flush-cache), 
that can visibly change the performance tradeoffs.

> I don't think the syntax-propertize mechanism is all that brilliant.
> It's too constrained, and places too many restrictions on what can be
> done with the syntax-table text property.  For example, (from
> syntax.el):

I wouldn't say it's perfect either. But it's proven helpful over the 
years, and provided a base design for a lot of major mode implementations.

> (defvar syntax-propertize-function nil
>    ;; Rather than a -functions hook, this is a -function because it's easier
>    ;; to do a single scan than several scans: with multiple scans, one cannot
>    ;; assume that the text before point has been propertized, so syntax-ppss
>    ;; gives unreliable results (and stores them in its cache to boot, so we'd
>    ;; have to flush that cache between each function, and we couldn't use
>    ;; syntax-ppss-flush-cache since that would not only flush the cache but also
>    ;; reset syntax-propertize--done which should not be done in this case).
>
>  From my point of view, "multiple scans" are _much_ easier.  They are
> prohibited here only because syntax-ppss and syntax-propertize-function
> have got themselves tied up in a tight knot.  One answer would be not to
> use syntax-ppss inside a s-p-function.  (CC Mode doesn't use syntax-ppss
> at all).  Another answer would be to give the responsibility of removing
> the s-t text properties to the s-p-function.

I think we could extend the customizability in that direction. But first 
we'd have to see clear evidence that the current design is not good 
enough (e.g. CC Mode has been reimplemented on top of it, and the result 
is decidedly not fast enough).

> > Simply collaborating with one other developer on an overhaul project
> > (whether it succeeds or not; perhaps partially) can improve on that.
>
> But take a massive amount of time.

You could consider it an investment.