Re: POC: customizable cc-mode keywords

[Alan Mackenzie]

Hi, Daniel.

Thanks for the detailed response.  In general, you're gradually
persuading me of your new approach.

On Sun, May 18, 2014 at 03:28:50PM -0700, Daniel Colascione wrote:
> On 05/18/2014 02:33 PM, Alan Mackenzie wrote:
> > On Fri, May 16, 2014 at 11:06:24AM -0700, Daniel Colascione wrote:
> >> On 05/16/2014 10:52 AM, Alan Mackenzie wrote:
> >>>>>> Today, we use clever macros to hard-code the values of all cc-mode
> >>>>>> language variables into the mode functions of each cc-mode major mode
> >>>>>> function or into c-init-language-vars-for, but in order to allow users
> >>>>>> to customize cc-mode syntax, we have to be able to recompute language
> >>>>>> constants and variables at runtime.

> >>>>> Do we, now?  You can imagine I've one or two reservations about this
> >>>>> idea. 

> >>>> What's your alternative?

> >>> Turning the pertinent c-lang-defvars, and only these, into configurable
> >>> variables in cc-vars.el.

> >> Have you looked into what task actually involves?

> > No I hadn't.  

> >> You need to modify c-decl-hangon-kwds, and as a result,
> >> c-prefix-spec-kwds, c-postfix-spec-kwds, c-keywords,
> >> c-keywords-regexp, c-keywords-obarray, c-nonlabel-token-key, and
> >> c-regular-keywords-regexp.

Thinking about it, changing the minimum number of c-lang-consts/vars to
the new scheme would also be a feasible strategy.  It would reduce the
opportunity of semi-experienced users to shoot themselves in the foot (as
we're discussing lower down).

> > OK.  Then another alternative is to use `or' forms in the appropriate
> > places, of which there are several rather than many, the extra arm of
> > the `or' looking something like "(looking-at c-noise-macro-re)".  Did you
> > consider this at all?  If so, why do you prefer the solution you now
> > propose?

> Doing it that way further increases the amount of work needed at parse
> time, and cc-mode isn't a speed demon already. Your proposal also
> introduces much-increased for complexity in the core and creates a risk
> of "missing a spot" and introducing a bug where we check the primary
> keywords but not the user-customized ones. (And what about the checking
> we do after calling intern-soft on the symbol obarray?) cc-engine is
> very difficult to follow as it is. I'd rather gracefully extend the
> existing functionality than add yet another set of special cases.

Yes, some slight loss of processing speed, and a little extra complexity.

> Another approach I was considering was to change cc-mode to always skip
> over regions marked with a certain text properly. Then a minor mode
> could independently parse and fontify the buffer, marking regions it
> wanted cc-mode to ignore with this property. This approach is more
> flexible, but I don't think the complexity is worth it when a simple
> extension to cc-mode is within reach.

I agree with you here.

[ .... ]

> > I've spent a lot of time trying to pin down exactly what your proposed
> > change does and how it does it, but amn't there yet.  I only have the
> > before and after versions, not a detailed description of the change.

> I'd hoped my initial message clearly explained the overall approach.
> Briefly, we want to allow users to tell cc-mode to ignore "noise macros"
> when parsing. IMHO, the best way to do that is to let user
> customizations augment the existing set of "noise macro" keywords
> cc-mode defines for each language. Today, cc-mode builds various data
> structures from these keyword sets at compile time. In order for user
> customizations to have any effect, we have to build these data
> structures at runtime instead. Doing that has no perceivable effect on
> mode initialization performance and lets user customizations have an effect.

I've got the overall strategy and motivation.  What I don't have is a
detailed description of the change - what defuns/defvars have been
changed, the nitty-gritty purpose of each change, the detailed
functionalities which have been removed/added, things like that.  In
short, what would go in a very detailed ChangeLog entry.  Diff files tend
to approach the "write-only" nature, and it's difficult to reconstruct
somebody else's thought patterns from only this diff file.

[ .... ]

> > One question which you haven't addressed yet is does the changed code
> > work properly on modes derived from a CC Mode mode?

Meaning, "Somebody" (tm) will need to test this.

> > How easy is it to create a `foo-extra-keywords' for such a mode?

> Trivial --- just include the variable in the c-lang-defconst calculation
> for the derived mode.

I don't follow the semantics here.  Hopefully maintainers of derived
modes won't feel the need to add extra c-lang-consts, since c-lang-consts
are meant to be defined for every language.  But answering my own
question, it should be easy enough to define `foo-extra-keywords' using
the same mechanism as for c-extra-keywords, since it will then be picked
up by the (c-mode-symbol "extra-keywords") in cc-langs.el.

> > Another is does the change work properly under XEmacs?

> I don't see why it wouldn't.

Well, it might be using, say, customize-... functions which don't (yet)
exist in XEmacs.  "Somebody" (tm) will need to test this too.

> > What is the purpose of {java,idl,pike}-extra-keywords, given that these
> > three languages don't have #defines?

> Consistency with the type defcustoms? Accommodations for weird custom
> preprocessors? Why not?

Why not, indeed?  Just there isn't one for AWK.  ;-)

> >> I don't agree, and I don't see anyone else proposing, much less
> >> implementing, a solution to this problem, .....

> > :-)  This problem was first highlighted as such, by you, this month, not
> > several years ago.

> There's been a TODO in the code to this effect since 2005 in the
> definition of c-decl-hangon-kwds; my patch removes this TODO.

Yes, OK.  But nobody's raised a fuss about it up till now.

> > Your patch is complicated, it's disruptive, yet you
> > seem put out that I want to discuss it and understand it, rather than
> > just blindly applying it as it is.  It seems your use of "You need to"
> > and "we have to" weren't mere hyperbole as I first thought.  Forgive me,
> > but I'm always suspicious when somebody tells me "there's no
> > alternative".  There are always alternatives.  The question is why is
> > your patch, which is not a straightforward or obvious change, the best
> > solution to the problem.  What are the tradeoffs?

> I discussed some of them above. The chief risk is that users might
> customize cc-mode by altering its language constants. I strongly
> discount this risk, especially since users can *already* do that by
> creating derived modes.

No, these activities are conceptually different.  Creating a derived mode
is clearly hacking, as contrasted with configuring.  The CC Mode language
constants are like defuns of typical major modes, not their configuration
variables.  It should somehow be clear to people changing the
c-lang-consts that they are actually hacking and not configuring.

[ .... ]

> >>> As I said, I'm not at all happy at making such a massive change to
> >>> CC Mode's architecture.  There would surely be unforeseen
> >>> consequences, some of which might well be negative.

> >> It's not a massive change in architecture.

> > As I said, I don't fully understand the change, yet.  But moving
> > calculations from compile time to run time _is_ a change of
> > architecture, and it seems to me not to be small.

> All the cc-defvar variables get the same values they did before, by
> default. cc-engine parses text the same way it always has. All the
> cc-mode customization remains. My patch merely lets users customize some
> existing cc-mode constants, then makes these customizations have runtime
> effect by removing some unnecessary old optimizations in cc-mode
> initialization. If you call that a massive change in architecture, then
> what I call architecture might as well be plate tectonics.

We've now reached the stage of arguing about words.  Let's stop!

> > It seems to me that the rigid separation between language (e.g. C)
> > and user configuration is being broken down.  How much easier will it
> > be for a user mistakenly to attempt to configure her CC Mode by
> > messing around with the c-lang-consts?

> Much easier --- but it's never been a priority in Emacs to prevent users
> from hanging themselves.

Yes, but who's going to have to bring the corpse down from the scaffold
and bury it?  By what mechanism will the said user come to understand
that c-lang-consts are not configuration variables?

> Users should be drawn to the customization settings we provide, and if
> they choose to bypass them, they probably know well enough what they're
> doing.

I've no problem with that if they're making an informed choice.  But even
in the current CC Mode mechanisms, I've occasionally been asked about
configuring CC Mode via c-lang-consts.  It shouldn't be too easy.

> As I mentioned, determined users can already derive from cc-mode
> built-in modes.

Yes.  But this is an intended facility, and isn't something that anyone
can stumble into by accident.

> >> cc-mode can already evaluate these variables at runtime in the case of
> >> a version mismatch.

> > That's a feature intended purely for CC Mode developers.

> Or for people who upgrade without recompiling derived modes. But it
> doesn't matter --- the dynamic-computation code is there and works fine.
> My change just uses it all the time instead of going down a different
> path involving dubious compile-time optimizations.

One other point.  Would it be possible to call foo-extra-keywords
something else?  What we're dealing with _aren't_ keywords like "if" or
"class", they're macros which expand to nothing.  I know that things like
"__attribute__" are currently called keywords, but extending the metaphor
to things like "__user" seems to be pushing things too far.  How about
"foo-noise-macros"?

-- 
Alan Mackenzie (Nuremberg, Germany).