Re: Handling extensions of programming languages

[Harald Jörg]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> [...]
>>   class Coffee::Machine extends Lawn::Mower
>>   {
>>      has $grinder :reader :writer(replace_grinder)
>>      method grind { ...; }
>>   } 
> [...]
>>  - add "Dishwasher" and "clean_up" to the imenu index.
>
> That seems to require AI (unless you're talking about a slightly
> different example than the one quoted above ;-).

Ouch.  I goofed up when deleting stuff from my test file to make the
example shorter :)  I wanted to add "grind" instead of "clean_up".

But, jokes aside: I actually consider adding entries to the imenu index
_which aren't there._ In the example above, Object::Pad will silently
create the methods `grinder' and `replace_grinder'.  I think these
*should* go to imenu because if your code in another source calls
$cm->grinder you might otherwise have a hard time finding where that
routine is declared.

>>  - make sure that indentation recognizes that the closing braces end a
>>    statement after "class" and "method".  Perl syntax has various cases
>>    where it doesn't.  I guess this is the part where SMIE would help.
>
> Actually, the closing brace which also closes a statement is one of the
> major pain points in `sm-c-mode`, so it would be one of the parts where
> you'd need extra work to make SMIE understand what's going on.

Given the effort CPerl mode spends to distinguish these two I guessed
so.  There are some open bugs regarding indentation in CPerl mode
(Bug#8077, Bug#11773, Bug#28640) which I'd like to fix while on the way.

Also, a few days ago there was a discussion in the Perlmonks forum where
CPerl mode guesses horribly wrong:
https://www.perlmonks.org/?node_id=11129870

>> For the latter two tasks, I need to "hook" the logic somehow into
>> CPerl's implementations of `imenu-create-index-function' and the various
>> indentation functions.  The current indentation code in CPerl mode
>> is... a bit messy, and some old bugs call for attention anyway.
>
> AFAIK font-lock and imenu are easy.  For font-lock there's
> `font-lock-add-keywords` and for imenu, you should be able to make it
> work fairly well with just `add-function` to
> `imenu-create-index-function`.

For certain values of easy :).  But yes, that's the plan.  The font-lock
mechanism is indeed very powerful.  For Object::Pad, the keyword
declaration takes about 120 lines (in rx notation, which is rather
wordy) - mostly due to the effort to avoid false positives.

For imenu, adding regexps to the list of `or'ed expressions to search
for seems to be an alternative.  Or maybe it doesn't, if I want to add
entries which can't be easily searched for.

> For indentation, it's fundamentally harder (for the same reason that
> combining two LALR grammars doesn't necessarily give you an LALR
> grammar), so it will have to be done in a somewhat ad-hoc way.

Indeed.  Indentation needs more "context".

> I suspect that if the base mode uses SMIE, it would make it
> significantly easier to add extensions (because the structure of SMIE
> imposes constraints that expose the "compositional" aspect of the
> grammar, in some sense), but that's not what you have to work with
> currently, so you're going to have to dig into the indentation code and
> try and figure out how to make it work with your extension(s) and then
> how to express the changes "from outside" (e.g. by using hooks,
> `add-function`, or `advice-add`; we can of course add hooks
> to `cperl-mode.el` or `perl-mode.el` to make that easier).

Your last parens touch another interesting aspect: Can that stuff be
used by cperl-mode.el _and_ perl-mode.el?

Well, as it turns out, the font-lock stuff "works" for both.  It looks a
bit weird with Perl mode because the "new" keywords like `method' have
different faces than the "old" ones like `my'.

For imenu, things are different: Perl mode uses
`imenu-generic-expression', whereas CPerl mode uses a rather complex
`imenu-create-index-function ', so that it can prepend the current
namespace to the name of functions.

And as for indentation...  I'd say the code in both modes needs to catch
up with current perl before we consider extensions.  Maybe they could
share functions or regular expressions how to find the beginning of a
function, or how to identify closing braces which terminate a statement:
The specification for this logic comes from Perl and should be the same
for both modes.
-- 
Cheers,
haj