Re: master 7362554: Widen around c-font-lock-fontify-region. This fixes bug #38049.

[Alan Mackenzie]

Hello Dmitry.

On Tue, Nov 12, 2019 at 15:36:08 +0200, Dmitry Gutov wrote:
> Hi Alan,

> On 11.11.2019 22:34, Alan Mackenzie wrote:

> > On Mon, Nov 11, 2019 at 18:52:34 +0200, Dmitry Gutov wrote:
> >> On 09.11.2019 16:40, Alan Mackenzie wrote:
> >>> +    (widen)

> >> Could you try and honor font-lock-dont-widen here?

> > Difficult.  In CC Mode it is sometimes necessary to enlarge a font
> > lock region so as to get the context needed to parse some element of
> > the buffer.  This was what gave rise to the bug.

> Did that really happen in the reported scenario? I mean,
> font-lock-dont-widen is usually nil, so font-lock will call widen for
> you already.

Cutting and pasting from the bug thread for bug #38049:

    [ Juri Linkov ]:
    > This is a reproducible test case:

    > 0. emacs -Q
    > 1. C-x C-f emacs/lib-src/emacsclient.c
    > 2. M-: (progn (search-forward "create-frame" nil t)
    > (reposition-window))

    > Then half screen displays unfontified lines.

    > Fontification doesn't fail in other modes, only in C mode.

    > This has something to do with interaction between c-font-lock
    > and buffer navigation in reposition-window.

    [ Alan Mackenzie ]:
    Indeed it does.

    Briefly,
    (i) reposition-window narrows to (2758 3940) in
      repos-count-screen-lines.
    (ii) This latter function uses vertical-motion to count the lines.
    (iii) vertical-motion triggers jit-lock fontification.
    (iv) This calls (eventually) c-font-lock-fontify-region.
    (v) c-font-lock-fontify-region attempts to examine buffer text before
      the start of the jit-lock chunk to find syntactic context.
    (vi) This is outside the visible region, so Emacs raises an exception.
    (vii) The exception is caught and discarded by an unwind-protect in
      c-font-lock-fontify-region.
    (viii) The jit-lock chunk remains unfontified.


> > In all of Emacs, there's just one use of font-lock-dont-widen, in
> > rmail.el, and that is almost certainly for a different reason than
> > your use of it (which is, I believe, having several major modes in a
> > single buffer).

> Does RMail use it for quoted pirces of code? If so, it would be
> similar.

I don't know RMail at all.  I imagine font-lock-dont-widen is used with
the region as the text of a mail message, excluding things like the
headers.  But I don't know.

> > Could you, perhaps, suggest some alternative to using widen in
> > c-font-lock-fontify-region?  Some way which would still work for CC
> > Mode (including the just fixed bug), but wouldn't adversely affect
> > the several major mode code you're maintaining?  Would it be
> > possible to widen "a bit" rather than widening to the whole buffer?
> > If so, how would I determine the bit to widen to?

> Not sure what difference the supposed "bit" would make (and thus, how 
> would mmm-mode determine it).

I was thinking about, perhaps, widening no wider than the CC Mode
portion of the several major mode buffer.

> But I think that would depend on your answer to my first question in
> this email. Because, I think, normally you don't "have to" widen at
> all, font-lock will do that for you.

I think it does, normally, but it didn't when the font locking was
invoked for vertical-motion.  (Font locking is needed to determine the
faces used, which might have different vertical sizes.)

> In mmm-mode context, however, we apply definite boundaries to the
> region chunks. Here's an example of some Noweb code:
> https://en.wikipedia.org/wiki/Noweb#Example_of_a_simple_noweb_program

> The inside of hello.c block would be narrowed to.

I think I've said this before, but I don't think narrowing is the right
tool for that task.  I don't think there is a suitable tool in Emacs at
the moment.

> Now, I have remembered that CC Mode calls widen from many places 
> already, so it already is problematic for using in a context like that. 

It does, yes.  Users also use widening and narrowing.

> Thus this particular change doesn't change the picture much, and so 
> please feel free to drop this conversation at any point of time.

OK.

-- 
Alan Mackenzie (Nuremberg, Germany).