[elpa] externals/parser-generator cee559d 139/434: Added separate document for lexical analysis documentation

[ELPA Syncer]

branch: externals/parser-generator
commit cee559dbf6545b21e296dc53de92e15239cfb5af
Author: Christian Johansson <christian@cvj.se>
Commit: Christian Johansson <christian@cvj.se>

    Added separate document for lexical analysis documentation
---
 README.md                |  2 +-
 docs/Lexical-Analysis.md | 29 +++++++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 7a87dbe..fa23e93 100644
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@ This is just started, so most stuff are *WIP*.
 
 ## Lexical Analysis
 
-We use a regular-language based lexical analyzer that can be implemented by a 
finite-state-machine (FSM). Set lexical analysis function by setting variable 
`parser-generator-lex-analyzer--function`. Optionally set reset function by 
setting variable `parser-generator-lex-analyzer--reset-function`. The lexical 
analysis is indexed on variable `parser-generator-lex-analyzer--index`. All 
parsers expect a list of tokens as response from lexical-analysis.
+We use a regular-language based lexical analyzer that can be implemented by a 
finite-state-machine (FSM). Read more [here](docs/Lexical-Analysis.md).
 
 ### Peek next look-ahead
 
diff --git a/docs/Lexical-Analysis.md b/docs/Lexical-Analysis.md
new file mode 100644
index 0000000..d648f87
--- /dev/null
+++ b/docs/Lexical-Analysis.md
@@ -0,0 +1,29 @@
+# Lexical Analysis
+
+Set lexical analysis function by setting variable 
`parser-generator-lex-analyzer--function`. Optionally set reset function by 
setting variable `parser-generator-lex-analyzer--reset-function`. The lexical 
analysis is indexed on variable `parser-generator-lex-analyzer--index`. All 
parsers expect a list of tokens as response from lexical-analysis.
+
+## Example
+
+``` emacs-lisp
+(setq
+   parser-generator-lex-analyzer--function
+   (lambda (index length)
+     (let* ((string '(("a" 1 . 2) ("b" 2 . 3) ("c" 3 . 4) ("d" 4 . 5)))
+            (string-length (length string))
+            (max-index (+ index length))
+            (tokens))
+       (while (and
+               (< index string-length)
+               (< index max-index))
+         (push (nth index string) tokens)
+     (setq index (1+ index)))
+(nreverse tokens))))
+(parser-generator-lex-analyzer--reset)
+(setq parser-generator--look-ahead-number 1)
+  (should
+   (equal
+    '(("a" 1 . 2))
+    (parser-generator-lex-analyzer--peek-next-look-ahead)))
+```
+
+[Back to start](../../../)