Re: "Font-lock is limited to text matching" is a myth

[Eric M. Ludlam]

Hi,

  CEDET, or more specifically, the Semantic parser/generator framework
uses font-lock for the following:

  * Pre-defined faces for colorizing text strings.
  * A simple test to see if point is in a comment or string.  (ie - not
code.)

  The concept of using the Semantic parser/generator framework for
improving font-locking accuracy has come up many times.  No-one to my
knowledge has attempted to mix the two.

  You can create a parser with the parser generator, and in the grammar
you create, put any kind of Lisp code you want.  You can also define
your own grammar-only macros.  (ie. there is a TAG macro for creating a
new tag at some point in a grammar.)  Thus, you could create a 'FONT'
macro, and use it to colorize a buffer while parsing the buffer.

  The PROS are that you could use a formal grammar to choose coloring
patterns.  That grammar could be shared by the tagging system.

  The CONS are that everything in Semantic is set up to parse the entire
buffer in one pass, and to parse logical sub-sections only after a full
parse has been done.

  Font lock/jit-lock is much better in the way it colorizes buffer
sections, and also the way it does so just before display (or so it
seems to me anyway) which you don't want to do w/ Semantic due to the
overhead of the parsing system.

  It probably DOES make sense to mix the two.  For example, font-lock
could refer to the tagging state to determine the logical bounds of an
edited entity, cutting down on partial expression matching problems.
Font lock could even be rigged up to just use the lexical analyzer for
some analysis if that makes sense.

  I would imagine that the parsing engine in Semantic, if it is deemed
critical by the maintainers, will get faster if key components are
integrated into the C code.  I would imagine a font-lock replacement
could be implemented fairly easily, but getting it to work as quickly as
font lock would take a fair effort.

  Lastly, as David Engster stated, CEDET has decoration tools that
decorate entire tags in some way, such as putting a line on top of
functions.  This is a separate decoration activity not related to font
lock, and something font lock would not be able to do reliably.

Enjoy
Eric


On Mon, 2009-08-10 at 22:51 +0200, Lennart Borgman wrote:
> On Mon, Aug 10, 2009 at 10:42 PM, David Engster<address@hidden> wrote:
> >> I think Xah meant the grammars. Maybe CEDET is not yet using
> >> font-lock, I do not know how old JIT-lock, which is an essential part
> >> for this, is.
> >
> > CEDET can create formatted strings from the tags, and it supports using
> > font-lock for that (see file semantic-format.el). There's also the
> > minor-mode 'semantic-decoration' (semantic-decorate.el) which applies
> > styles to certain tags in the buffer while editing, and this mode does
> > not use font-lock.
> 
> 
> Thanks David. What is the reason that font-lock is not used there?
> 
> I do not know much about it, but am a bit curious. Does CEDET use the
> font-lock/JIT-lock frame work for handling parsing requirements?
> 
> 
> > Since Semantic usually does not parse every
> > expression in the buffer, this decoration is currently limited to stuff
> > like include statements, function declarations or protected class
> > members. (Eric, please correct me if I'm wrong here).
> >
> > -David
> >