Search-based Fontification.

[Lute Kamstra]

I've updated the node Search-based Fontification in the lisp manual.
I also adapted the docstring of font-lock-keywords.  Ok to commit?

Lute.


2005-04-06  Lute Kamstra  <address@hidden>

        * modes.texi (Search-based Fontification): Fix cross references.
        Use consistent terminology.  Document anchored highlighting.

Index: lispref/modes.texi
===================================================================
RCS file: /cvsroot/emacs/emacs/lispref/modes.texi,v
retrieving revision 1.94
diff -c -r1.94 modes.texi
*** lispref/modes.texi  5 Apr 2005 15:21:56 -0000       1.94
--- lispref/modes.texi  6 Apr 2005 15:24:29 -0000
***************
*** 1,6 ****
  @c -*-texinfo-*-
  @c This is part of the GNU Emacs Lisp Reference Manual.
! @c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999, 
  @c   2003, 2004, 2005 Free Software Foundation, Inc.
  @c See the file elisp.texi for copying conditions.
  @setfilename ../info/modes
--- 1,6 ----
  @c -*-texinfo-*-
  @c This is part of the GNU Emacs Lisp Reference Manual.
! @c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999,
  @c   2003, 2004, 2005 Free Software Foundation, Inc.
  @c See the file elisp.texi for copying conditions.
  @setfilename ../info/modes
***************
*** 2111,2117 ****
  each element, it finds and handles all matches.  Ordinarily, once
  part of the text has been fontified already, this cannot be overridden
  by a subsequent match in the same text; but you can specify different
! behavior using the @var{override} element of a @var{highlighter}.
  
    Each element of @code{font-lock-keywords} should have one of these
  forms:
--- 2111,2117 ----
  each element, it finds and handles all matches.  Ordinarily, once
  part of the text has been fontified already, this cannot be overridden
  by a subsequent match in the same text; but you can specify different
! behavior using the @var{override} element of a @var{subexp-highlighter}.
  
    Each element of @code{font-lock-keywords} should have one of these
  forms:
***************
*** 2122,2135 ****
  @code{font-lock-keyword-face}.  For example,
  
  @example
! ;; @r{Highlight discrete occurrences of @samp{foo}}
  ;; @r{using @code{font-lock-keyword-face}.}
  "\\<foo\\>"
  @end example
  
! The function @code{regexp-opt} (@pxref{Syntax of Regexps}) is useful for
! calculating optimal regular expressions to match a number of different
! keywords.
  
  @item @var{function}
  Find text by calling @var{function}, and highlight the matches
--- 2122,2135 ----
  @code{font-lock-keyword-face}.  For example,
  
  @example
! ;; @r{Highlight occurrences of the word @samp{foo}}
  ;; @r{using @code{font-lock-keyword-face}.}
  "\\<foo\\>"
  @end example
  
! The function @code{regexp-opt} (@pxref{Regexp Functions}) is useful
! for calculating optimal regular expressions to match a number of
! different keywords.
  
  @item @var{function}
  Find text by calling @var{function}, and highlight the matches
***************
*** 2146,2155 ****
  @var{function} fails.  On failure, @var{function} need not reset point
  in any particular way.
  
! @item (@var{matcher} . @var{match})
  In this kind of element, @var{matcher} is either a regular
  expression or a function, as described above.  The @sc{cdr},
! @var{match}, specifies which subexpression of @var{matcher} should be
  highlighted (instead of the entire text that @var{matcher} matched).
  
  @example
--- 2146,2155 ----
  @var{function} fails.  On failure, @var{function} need not reset point
  in any particular way.
  
! @item (@var{matcher} . @var{subexp})
  In this kind of element, @var{matcher} is either a regular
  expression or a function, as described above.  The @sc{cdr},
! @var{subexp}, specifies which subexpression of @var{matcher} should be
  highlighted (instead of the entire text that @var{matcher} matched).
  
  @example
***************
*** 2159,2166 ****
  @end example
  
  If you use @code{regexp-opt} to produce the regular expression
! @var{matcher}, then you can use @code{regexp-opt-depth} (@pxref{Syntax
! of Regexps}) to calculate the value for @var{match}.
  
  @item (@var{matcher} . @var{facespec})
  In this kind of element, @var{facespec} is an object which specifies
--- 2159,2166 ----
  @end example
  
  If you use @code{regexp-opt} to produce the regular expression
! @var{matcher}, then you can use @code{regexp-opt-depth} (@pxref{Regexp
! Functions}) to calculate the value for @var{subexp}.
  
  @item (@var{matcher} . @var{facespec})
  In this kind of element, @var{facespec} is an object which specifies
***************
*** 2173,2209 ****
  ("fubar" . fubar-face)
  @end example
  
! However, @var{facespec} can also be a list of the form
  
  @example
  (face @var{face} @var{prop1} @var{val1} @var{prop2} @address@hidden)
  @end example
  
! to specify various text properties to put on the text that matches.
! If you do this, be sure to add the other text property names that you
! set in this way to the value of @code{font-lock-extra-managed-props}
! so that the properties will also be cleared out when they are no longer
! appropriate.
  
! @item (@var{matcher} . @var{highlighter})
! In this kind of element, @var{highlighter} is a list
  which specifies how to highlight matches found by @var{matcher}.
! It has the form
  
  @example
! (@var{subexp} @var{facespec} @var{override} @var{laxmatch})
  @end example
  
  The @sc{car}, @var{subexp}, is an integer specifying which subexpression
  of the match to fontify (0 means the entire matching text).  The second
  subelement, @var{facespec}, specifies the face, as described above.
  
! The last two values in @var{highlighter}, @var{override} and
! @var{laxmatch}, are flags.  If @var{override} is @code{t}, this
! element can override existing fontification made by previous elements
! of @code{font-lock-keywords}.  If it is @code{keep}, then each
! character is fontified if it has not been fontified already by some
! other element.  If it is @code{prepend}, the face specified by
  @var{facespec} is added to the beginning of the @code{font-lock-face}
  property.  If it is @code{append}, the face is added to the end of the
  @code{font-lock-face} property.
--- 2173,2211 ----
  ("fubar" . fubar-face)
  @end example
  
! However, @var{facespec} can also be a list of the form:
  
  @example
  (face @var{face} @var{prop1} @var{val1} @var{prop2} @address@hidden)
  @end example
  
! to specify the face @var{face} and various additional text properties
! to put on the text that matches.  If you do this, be sure to add the
! other text property names that you set in this way to the value of
! @code{font-lock-extra-managed-props} so that the properties will also
! be cleared out when they are no longer appropriate.  Alternatively,
! you can set the variable @code{font-lock-unfontify-region-function} to
! a function that clears these properties.
  
! @item (@var{matcher} . @var{subexp-highlighter})
! In this kind of element, @var{subexp-highlighter} is a list
  which specifies how to highlight matches found by @var{matcher}.
! It has the form:
  
  @example
! (@var{subexp} @var{facespec} address@hidden address@hidden)
  @end example
  
  The @sc{car}, @var{subexp}, is an integer specifying which subexpression
  of the match to fontify (0 means the entire matching text).  The second
  subelement, @var{facespec}, specifies the face, as described above.
  
! The last two values in @var{subexp-highlighter}, @var{override} and
! @var{laxmatch}, are optional flags.  If @var{override} is @code{t},
! this element can override existing fontification made by previous
! elements of @code{font-lock-keywords}.  If it is @code{keep}, then
! each character is fontified if it has not been fontified already by
! some other element.  If it is @code{prepend}, the face specified by
  @var{facespec} is added to the beginning of the @code{font-lock-face}
  property.  If it is @code{append}, the face is added to the end of the
  @code{font-lock-face} property.
***************
*** 2219,2226 ****
  Here are some examples of elements of this kind, and what they do:
  
  @smallexample
! ;; @r{Highlight occurrences of either @samp{foo} or @samp{bar},}
! ;; @r{using @code{foo-bar-face}, even if they have already been highlighted.}
  ;; @address@hidden should be a variable whose value is a face.}
  ("foo\\|bar" 0 foo-bar-face t)
  
--- 2221,2228 ----
  Here are some examples of elements of this kind, and what they do:
  
  @smallexample
! ;; @r{Highlight occurrences of either @samp{foo} or @samp{bar}, using}
! ;; @address@hidden, even if they have already been highlighted.}
  ;; @address@hidden should be a variable whose value is a face.}
  ("foo\\|bar" 0 foo-bar-face t)
  
***************
*** 2230,2293 ****
  (fubar-match 1 fubar-face)
  @end smallexample
  
! @item (@var{matcher} @address@hidden)
! This sort of element specifies several @var{highlighter} lists for a
! single @var{matcher}.  In order for this to be useful, each
! @var{highlighter} should have a different value of @var{subexp}; that is,
! each one should apply to a different subexpression of @var{matcher}.
! 
! @ignore
! @item (@var{matcher} . @var{anchored})
! In this kind of element, @var{anchored} acts much like a
! @var{highlighter}, but it is more complex and can specify multiple
! successive searches.
! 
! For highlighting single items, typically only @var{highlighter} is
! required.  However, if an item or (typically) items are to be
! highlighted following the instance of another item (the anchor) then
! @var{anchored} may be required.
  
! It has this format:
  
! @example
! (@var{submatcher} @var{pre-match-form} @var{post-match-form} @address@hidden)
! @end example
! 
! @c I can't parse this text -- rms
! where @var{submatcher} is much like @var{matcher}, with one
! exception---see below.  @var{pre-match-form} and @var{post-match-form}
! are evaluated before the first, and after the last, instance
! @var{anchored}'s @var{submatcher} is used.  Therefore they can be used
! to initialize before, and cleanup after, @var{submatcher} is used.
! Typically, @var{pre-match-form} is used to move to some position
! relative to the original @var{submatcher}, before starting with
! @var{anchored}'s @var{submatcher}.  @var{post-match-form} might be used
! to move, before resuming with @var{anchored}'s parent's @var{matcher}.
  
! For example, an element of the form highlights (if not already highlighted):
  
! @example
! ("\\<anchor\\>" (0 anchor-face) ("\\<item\\>" nil nil (0 item-face)))
! @end example
  
! Discrete occurrences of @samp{anchor} in the value of
! @code{anchor-face}, and subsequent discrete occurrences of @samp{item}
! (on the same line) in the value of @code{item-face}.  (Here
! @var{pre-match-form} and @var{post-match-form} are @code{nil}.
! Therefore @samp{item} is initially searched for starting from the end of
! the match of @samp{anchor}, and searching for subsequent instance of
! @samp{anchor} resumes from where searching for @samp{item} concluded.)
  
! The above-mentioned exception is as follows.  The limit of the
! @var{submatcher} search defaults to the end of the line after
! @var{pre-match-form} is evaluated.  However, if @var{pre-match-form}
! returns a position greater than the position after @var{pre-match-form}
! is evaluated, that position is used as the limit of the search.  It is
! generally a bad idea to return a position greater than the end of the
! line; in other words, the @var{submatcher} search should not span lines.
  
! @item (@var{matcher} @var{highlighters-or-anchoreds} ...)
! @end ignore
  
  @item (eval . @var{form})
  Here @var{form} is an expression to be evaluated the first time
--- 2232,2303 ----
  (fubar-match 1 fubar-face)
  @end smallexample
  
! @item (@var{matcher} . @var{anchored-highlighter})
  
! In this kind of element, @var{anchored-highlighter} specifies how to
! highlight text that follows a match found by @var{matcher}.  So a
! match found by @var{matcher} acts as the anchor for further searches
! specified by @var{anchored-highlighter}.  @var{anchored-highlighter}
! is a list of the following form:
! 
! @example
! (@var{anchored-matcher} @var{pre-form} @var{post-form} 
!                         @address@hidden)
! @end example
! 
! Here, @var{anchored-matcher}, like @var{matcher}, is either a regular
! expression or a function.  After a match of @var{matcher} is found,
! point is at the end of the match and the form @var{pre-form} is
! evaluated.  Then Font Lock searches for matches of
! @var{anchored-matcher} and uses @var{subexp-highlighters} to highlight
! these.  A @var{subexp-highlighter} is as described above.  Finally,
! @var{post-form} is evaluated.
! 
! The forms @var{pre-form} and @var{post-form} can be used to initialize
! before, and cleanup after, @var{anchored-matcher} is used.  Typically,
! @var{pre-form} is used to move point to some position relative to the
! match of @var{matcher}, before starting with @var{anchored-matcher}.
! @var{post-form} might be used to move back, before resuming with
! @var{matcher}.
! 
! After @var{pre-form} is evaluated, Font Lock does not search for
! @var{anchored-matcher} beyond the end of the line.  However, if
! @var{pre-form} returns a buffer position that is greater than the
! position of point after @var{pre-form} is evaluated, then the position
! returned by @var{pre-form} is used as the limit of the search instead.
! It is generally a bad idea to return a position greater than the end
! of the line; in other words, the @var{anchored-matcher} search should
! not span lines.
  
! For example,
  
! @smallexample
! ;; @r{Highlight occurrences of the word @samp{item} following}
! ;; @r{an occurrence of the word @samp{anchor} (on the same line)}
! ;; @r{in the value of @code{item-face}.}
! ("\\<anchor\\>" "\\<item\\>" nil nil (0 item-face))
! @end smallexample
  
! Here, @var{pre-form} and @var{post-form} are @code{nil}.  Therefore
! @samp{item} is initially searched for starting from the end of the
! match of @samp{anchor}, and searching for subsequent instances of
! @samp{anchor} resumes from where searching for @samp{item} concluded.
  
! @item (@var{matcher} @address@hidden)
! This sort of element specifies several @var{highlighter} lists for a
! single @var{matcher}.  A @var{highlighter} list can be of the type
! @var{subexp-highlighter} or @var{anchored-highlighter} as described
! above.
  
! For example,
  
! @smallexample
! ;; @r{Highlight occurrences of the word @samp{anchor} in the value}
! ;; @r{of @code{anchor-face}, and subsequent occurrences of the word}
! ;; @address@hidden (on the same line) in the value of @code{item-face}.}
! ("\\<anchor\\>" (0 anchor-face)
!                 ("\\<item\\>" nil nil (0 item-face)))
! @end smallexample
  
  @item (eval . @var{form})
  Here @var{form} is an expression to be evaluated the first time
***************
*** 2301,2307 ****
  updating when you edit the buffer does not, since it considers text one
  line at a time.  If you have patterns that typically only span one
  line but can occasionally span two or three, such as
! @samp{<title>...</title>}, you can ask font-lock to be more careful by
  setting @code{font-lock-multiline} to @code{t}.  But it still will not
  work in all cases.
  
--- 2311,2317 ----
  updating when you edit the buffer does not, since it considers text one
  line at a time.  If you have patterns that typically only span one
  line but can occasionally span two or three, such as
! @samp{<title>...</title>}, you can ask Font Lock to be more careful by
  setting @code{font-lock-multiline} to @code{t}.  But it still will not
  work in all cases.
  


Index: lisp/font-lock.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/font-lock.el,v
retrieving revision 1.236
diff -c -r1.236 font-lock.el
*** lisp/font-lock.el   28 Mar 2005 16:45:43 -0000      1.236
--- lisp/font-lock.el   6 Apr 2005 15:33:42 -0000
***************
*** 383,396 ****
  
  MATCH-HIGHLIGHT should be of the form:
  
!  (MATCH FACENAME OVERRIDE LAXMATCH)
  
  MATCH is the subexpression of MATCHER to be highlighted.  FACENAME is an
  expression whose value is the face name to use.  Face default attributes
  can be modified via \\[customize].  Instead of a face, FACENAME can
! evaluate to a property list of the form (face VAL1 PROP2 VAL2 PROP3 VAL3 ...)
  in which case all the listed text-properties will be set rather than
! just `face'.  In such a case, you will most likely want to put those
  properties in `font-lock-extra-managed-props' or to override
  `font-lock-unfontify-region-function'.
  
--- 383,396 ----
  
  MATCH-HIGHLIGHT should be of the form:
  
!  (MATCH FACENAME [[OVERRIDE [LAXMATCH]])
  
  MATCH is the subexpression of MATCHER to be highlighted.  FACENAME is an
  expression whose value is the face name to use.  Face default attributes
  can be modified via \\[customize].  Instead of a face, FACENAME can
! evaluate to a property list of the form (face FACE PROP1 VAL1 PROP2 VAL2 ...)
  in which case all the listed text-properties will be set rather than
! just FACE.  In such a case, you will most likely want to put those
  properties in `font-lock-extra-managed-props' or to override
  `font-lock-unfontify-region-function'.
  
***************
*** 426,432 ****
  used to initialise before, and cleanup after, MATCHER is used.  Typically,
  PRE-MATCH-FORM is used to move to some position relative to the original
  MATCHER, before starting with MATCH-ANCHORED's MATCHER.  POST-MATCH-FORM might
! be used to move, before resuming with MATCH-ANCHORED's parent's MATCHER.
  
  For example, an element of the form highlights (if not already highlighted):
  
--- 426,432 ----
  used to initialise before, and cleanup after, MATCHER is used.  Typically,
  PRE-MATCH-FORM is used to move to some position relative to the original
  MATCHER, before starting with MATCH-ANCHORED's MATCHER.  POST-MATCH-FORM might
! be used to move back, before resuming with MATCH-ANCHORED's parent's MATCHER.
  
  For example, an element of the form highlights (if not already highlighted):
  
***************
*** 436,442 ****
   discrete occurrences of \"item\" (on the same line) in the value of 
`item-face'.
   (Here PRE-MATCH-FORM and POST-MATCH-FORM are nil.  Therefore \"item\" is
   initially searched for starting from the end of the match of \"anchor\", and
!  searching for subsequent instance of \"anchor\" resumes from where searching
   for \"item\" concluded.)
  
  The above-mentioned exception is as follows.  The limit of the MATCHER search
--- 436,442 ----
   discrete occurrences of \"item\" (on the same line) in the value of 
`item-face'.
   (Here PRE-MATCH-FORM and POST-MATCH-FORM are nil.  Therefore \"item\" is
   initially searched for starting from the end of the match of \"anchor\", and
!  searching for subsequent instances of \"anchor\" resumes from where searching
   for \"item\" concluded.)
  
  The above-mentioned exception is as follows.  The limit of the MATCHER search