bug#16526: 24.3.50; scroll-conservatively & c-mode regression

[martin rudalics]

> No.  The particular rogue invocation of scan-lists which caused this
> (sub-)thread scans from BOB to near EOB 29 times.  Inside scan-lists,
> _NO_ changes to syntax-tables, s-t text properties etc. are possible.
> What I'm suggesting is that cacheing that first scan from BOB will be a
> big win for many time-consuming scan-listses.

To obtain a meaningful result from `scan-lists' with a negative COUNT,
we have to know whether the starting point is outside a comment or
string.  This means that we either first have to scan from BOB till that
point or have the results from a previous scan in a not yet invalidated
cache anyway.  Can we agree on that?

Now it's possible that CC-mode for some magic reason knows that the
starting point meets the requirement above.  In that case the magic used
should also assure that `scan-lists' stops at a reasonable position (one
known to have DEPTH zero when scanned from BOB) by narrowing the buffer
appropriately.

Otherwise we need some heuristics to provide a reasonable amount of
defun starters to be eventually used by back_comment (I suppose it's
that routine causing our problems).  In any case, these starters must be
provided by `parse-partial-sexp' (or forward_sexp as you suggest).  I
suppose we agree on that.

In any case the question is why any defun starters cached before
invoking `scan-lists' would be invalid.  I can't see a good reason.  Can
you give me one?.

>> And how would a defun start cache handle such constructs?
>
> Such a cache would be initialised to nil within a particular invocation
> of scan-lists, hence nothing like the above could cause any trouble.

It's a waste that two subsequent `scan-lists' would not share the same
cache but let's stay with this assumption for the rest.

> scan-lists is an atomic function, without memory of any state.  It's
> functionality is wholly determined by the current buffer state and the
> current setting of (quite a few) state variables.
>
> syntax-ppss is motivated by cacheing historical state, hence is
> critically different.

Then we have to flush its cache when calling `scan-lists'.

> OK, I think I see what you're getting at.  You're thinking of a cache
> which would endure over random buffer changes.  The cache I was intending
> would be flushed at the beginning of each scan-lists.  "What I tell you
> three times is true." (Lewis Carroll, The Hunting of the Snark).

I was thinking of one cache per buffer that would be produced by parsing
from BOB.  A second cache would mean to double the overhead for writing
and maintaining the respective code.

> It's truncated at buffer changes.  It doesn't contain any information
> which could become stale on any of the other sorts of modifications we've
> been talking about, or it isn't used when they could hurt anything.  That
> this cache has a tightly defined purpose is why it can be robust, whereas
> syntax-ppss's cache, being very general purpose, is vulnerable to certain
> changes.

Which obviously means that we'd have to improve the robustness of the
syntax-ppss cache.

>> What you're asking for is impossible:
>
>> (1) You want to base find_defun_start on scan_lists starting at BOB.
>
> No, on scan_sexps_forward, which (despite the confusing name) is the core
> of parse-partial-sexp without its lispy arguments.

OK

>>       This means that you need a function that repeatedly calls
>>       scan_lists, stopping whenever depth reaches zero and remembers that
>>       position, returning the last such position before FROM.  Such a
>>       function exists already - it's called `parse-partial-sexp'.
>
> Yes.  Or scan_sexps_forward which is the C core of it.

OK

>> (2) You want to avoid that function call scan_lists repeatedly by
>>       caching previously found positions.  Such a function exists already
>>       - it's called `syntax-ppss'.
>
> I'd have nothing against using syntax-ppss, as long as its cache was
> bound to nil for each scan-lists call - except it would cause some
> slowdown (compared with a special-purpose cache written in C), since it
> goes through the lisp bytecode interpreter.

Then we have to provide a C-level interface to that cache which can be
used by find_defun_start.

>> (3) You want that function to work even if someone changes the syntax
>>       table or disables `before-change-functions' without flushing its
>>       cache.  Such a function does not exist and never will.
>
> No, I want that cache to be flushed for each scan-lists call, so that the
> first scan of (nearly) the whole buffer will create the cache, and the
> other 28 scans will use that cache.

This means that we'd have to specify which defun starts to store in that
cache and replace the `open-paren-in-column-0-is-defun-start' stuff in
find_defun_start by going to the last position stored in that cache.
The latter should be trivial but is certainly not suited for Emacs 24.4.
The former is less trivial (IMHO) and requires a general solution within
the context of `syntax-ppss' which might even mean to move the cache
building and flushing parts of the latter to C.

martin