Re: Regarding Emacs, js.el, template-strings and syntax-tables

[Jostein Kjønigsen]

On Thu, Aug 24, 2017, at 12:53 PM, Stefan Monnier wrote:

Indeed this problem has been with us in sh-script.el for many years.

There are several different aspects at play:

- how should font-lock display `name`.  Should it have a normal face as

  if it weren't inside a string?  Or should have a kind of combination

  of string-face and something else?

I'd vote for "plain" face, and name being styled as it would have been outside the string-context. If a mode-author wants to fancy it up, we should definitely have a way to easily accommodate that though.

Having a separate _expression_-in-string-or-comment face is probably the best approach in that regard.

- how should we handle nestings of such constructs.

That's a good question, but I think a even better question would be to ask what that would look like, and if we should strive to support it in the first place.

The most contrived (again JS) example I can think of would be inline code-references in a comment in a _expression_ in a string. Something like:

    let greeting = "Hello, ${ /** @name string here refers to  the employer, not the employee! */ employee_name }";

If we were to support these kind of endless constructs (which I'm sure someone will find a way to make), we would need to consider this a level of nesting, just like we do with parens and curly braces and what not.

- several parts of Emacs like to know if we're inside

  code/string/comment (usually using (nth [3|4|8] (syntax-ppss))).

  Should these all consider "name" as being inside code?

  Or inside string?  Or should we revisit all those cases one-by-one?

It's hard to come up with a blanket solution for this, just like that. I definitely recognize the problem though.

In increasing the expressiveness of the syntax-construction, it's generally hard to say "all new occurrences of this new construct map perfectly to this old understanding of the same property" (as in the Liskov Substitution Principle).

If a change means revisiting cases one by one, the Emacs ecosystem will be in a state of mess for years to come, so that's clearly not an option.

How about instead extending (syntax-ppss) to keep all existing behaviour for all existing properties, but include some new ones with more information? Then mode-authors which wants to exploit these new properties can probe for those in a reasonable backwards-compatible manner?

Ideally these extra properties should contain at least the following:

* is in escape string or comment

* start of this-level string/comment escape-sequence (nth 8 contains the start of the string/comment)

* level of string/comment-_expression_ nesting

As far as I can see, that should about cover it.

In sh-script, we use syntax-propertize to make sure the whole string and

its contents is all considered as a string and nothing else.  This was

the simplest solution and probably corresponds to the current behavior

of js.el (and typescript.el) as well.

While obviously the simplest, it's syntactically inaccurate and this inaccuracy is causing issues in other packages who rely on accurate syntax-information.

It would be really nice if we could have our cake and eat it too.

--

Jostein Kjønigsen

address@hidden 🍵 address@hidden

https://jostein.kjonigsen.net