[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[elpa] externals/parser-generator 1879cb0 271/434: Merge branch 'master'
From: |
ELPA Syncer |
Subject: |
[elpa] externals/parser-generator 1879cb0 271/434: Merge branch 'master' of git.cvj.se:/home/git/emacs-parser-generator |
Date: |
Mon, 29 Nov 2021 15:59:55 -0500 (EST) |
branch: externals/parser-generator
commit 1879cb07dc55834323a5b3d82e896324d813f2d1
Merge: 8e3084b 1b9d8db
Author: Christian Johansson <christian@cvj.se>
Commit: Christian Johansson <christian@cvj.se>
Merge branch 'master' of git.cvj.se:/home/git/emacs-parser-generator
---
README.md | 2 +-
docs/Lexical-Analysis.md | 90 ++++++++++++++++++------------
docs/Syntax-Analysis/LRk.md | 14 ++++-
test/parser-generator-lex-analyzer-test.el | 1 +
4 files changed, 66 insertions(+), 41 deletions(-)
diff --git a/README.md b/README.md
index 5e7c233..cdc924c 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
[![License GPL
3](https://img.shields.io/badge/license-GPL_3-green.svg)](https://www.gnu.org/licenses/gpl-3.0.txt)
[![Build
Status](https://travis-ci.org/cjohansson/emacs-parser-generator.svg?branch=master)](https://travis-ci.org/cjohansson/emacs-parser-generator)
-The idea of this plugin is to provide functions for various kinds of
context-free grammar parser generations with support for
syntax-directed-translations (SDT) and semantic actions (SA). This project is
about implementing algorithms described in the book `The Theory of Parsing,
Translation and Compiling (Volume 1)` by `Alfred V. Aho and Jeffrey D. Ullman`
(1972). Also this project is about me learning how to parse languages.
+The idea of this plugin is to provide functions for various kinds of
context-free grammar parser generations with support for
syntax-directed-translations (SDT) and semantic actions (SA) and the
possibility of exporting parsers and translators to enable plugin-agnostic
usage. This project is also about implementing algorithms described in the book
`The Theory of Parsing, Translation and Compiling (Volume 1)` by `Alfred V. Aho
and Jeffrey D. Ullman` (1972). Also this project is about me l [...]
This is just started, so most stuff are *WIP*.
diff --git a/docs/Lexical-Analysis.md b/docs/Lexical-Analysis.md
index 45f4b8c..27d0722 100644
--- a/docs/Lexical-Analysis.md
+++ b/docs/Lexical-Analysis.md
@@ -2,74 +2,90 @@
Set lexical analysis function by setting variable
`parser-generator-lex-analyzer--function`. Optionally set reset function by
setting variable `parser-generator-lex-analyzer--reset-function`. The lexical
analysis is indexed on variable `parser-generator-lex-analyzer--index`. All
parsers expect a list of tokens as response from lexical-analysis.
-### Peek next look-ahead
+## Token
-Returns the look-ahead number of next terminals in stream.
+A token is defined as a list with 3 elements, first is a string or symbol,
second is the start index of token in stream and third is the end index of
token in stream, second and third element have a dot between them, this
structure is to be compatible with Emacs Semantic system. Example token:
+
+``` emacs-lisp
+'("a" 1 . 2)
+```
+
+## Peek next look-ahead
+
+Returns the look-ahead number of next terminals in stream, if end of stream is
reached a EOF-identifier is returned. Result is expected to be a list with each
token in it.
``` emacs-lisp
(require 'ert)
(setq
parser-generator-lex-analyzer--function
- (lambda (index length)
- (let* ((string '(a a b b b))
+ (lambda (index)
+ (let* ((string '(("a" 1 . 2) ("b" 2 . 3) ("c" 3 . 4) ("d" 4 . 5)))
(string-length (length string))
- (max-index (+ index length))
- (tokens))
+ (max-index index)
+ (tokens)
+ (next-token))
(while (and
- (< index string-length)
- (< index max-index))
- (push (nth index string) tokens)
+ (< (1- index) string-length)
+ (< (1- index) max-index))
+ (setq next-token (nth (1- index) string))
+ (push next-token tokens)
(setq index (1+ index)))
(nreverse tokens))))
(parser-generator-lex-analyzer--reset)
(setq parser-generator--look-ahead-number 1)
- (should
- (equal
- '(a)
- (parser-generator-lex-analyzer--peek-next-look-ahead)))
+(should
+ (equal
+ '(("a" 1 . 2))
+ (parser-generator-lex-analyzer--peek-next-look-ahead)))
- (setq parser-generator--look-ahead-number 2)
- (should
- (equal
- '(a b)
- (parser-generator-lex-analyzer--peek-next-look-ahead)))
+(setq parser-generator--look-ahead-number 2)
+(should
+ (equal
+ '(("a" 1 . 2) ("b" 2 . 3))
+ (parser-generator-lex-analyzer--peek-next-look-ahead)))
+(setq parser-generator--look-ahead-number 10)
+(should
+ (equal
+ '(("a" 1 . 2) ("b" 2 . 3) ("c" 3 . 4) ("d" 4 . 5) ($) ($) ($) ($) ($) ($))
+ (parser-generator-lex-analyzer--peek-next-look-ahead)))
```
-### Pop token
+## Pop token
-Returns the next token in stream and moves the index one point forward.
+Returns the next token in stream and moves the lexical analyzer index one
point forward. If end of stream is reached return nil. The result is expected
to be a list containing each token popped.
``` emacs-lisp
(require 'ert)
(setq
parser-generator-lex-analyzer--function
- (lambda (index length)
- (let* ((string '(a b))
+ (lambda (index)
+ (let* ((string '(("a" 1 . 2) ("b" 2 . 3)))
(string-length (length string))
- (max-index (+ index length))
+ (max-index index)
(tokens))
(while (and
- (< index string-length)
- (< index max-index))
- (push (nth index string) tokens)
+ (< (1- index) string-length)
+ (< (1- index) max-index))
+ (push (nth (1- index) string) tokens)
(setq index (1+ index)))
(nreverse tokens))))
(parser-generator-lex-analyzer--reset)
+(setq parser-generator--look-ahead-number 1)
+(should
+ (equal
+ '(("a" 1 . 2))
+ (parser-generator-lex-analyzer--pop-token)))
+(should
+ (equal
+ '(("b" 2 . 3))
+ (parser-generator-lex-analyzer--pop-token)))
(should
- (equal
- '(a)
- (parser-generator-lex-analyzer--pop-token)))
- (should
- (equal
- '(b)
- (parser-generator-lex-analyzer--pop-token)))
- (should
- (equal
- nil
- (parser-generator-lex-analyzer--pop-token)))
+ (equal
+ nil
+ (parser-generator-lex-analyzer--pop-token)))
```
[Back to start](../../../)
diff --git a/docs/Syntax-Analysis/LRk.md b/docs/Syntax-Analysis/LRk.md
index 04c8d59..cc11a9a 100644
--- a/docs/Syntax-Analysis/LRk.md
+++ b/docs/Syntax-Analysis/LRk.md
@@ -100,7 +100,10 @@ Each production RHS can optionally contain a
lambda-expression that will be call
(switch-to-buffer buffer)
(insert "if (a) { b; }")
- (parser-generator-set-grammar '((Sp S) (";" OPEN_ROUND_BRACKET
CLOSE_ROUND_BRACKET ECHO IF OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE)
((Sp S) (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET VARIABLE ";" CLOSE_CURLY_BRACKET (lambda(args) (format
"(when %s %s)" (nth 2 args) (nth 5 args)))))) Sp))
+ (parser-generator-set-grammar
+ '((Sp S) (";" OPEN_ROUND_BRACKET CLOSE_ROUND_BRACKET ECHO IF
OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE)
+ ((Sp S) (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET VARIABLE ";"
+ CLOSE_CURLY_BRACKET (lambda(args) (format "(when %s %s)" (nth 2
args) (nth 5 args)))))) Sp))
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -153,7 +156,12 @@ Each production RHS can optionally contain a
lambda-expression that will be call
(switch-to-buffer buffer)
(kill-region (point-min) (point-max))
- (parser-generator-set-grammar '((Sp S T) (";" OPEN_ROUND_BRACKET
CLOSE_ROUND_BRACKET ECHO IF OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE)
((Sp S) (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET T CLOSE_CURLY_BRACKET (lambda(args) (format "(when %s %s)"
(nth 2 args) (nth 5 args))))) (T (ECHO VARIABLE ";" (lambda(args) (format
"(message %s)" (nth 1 args)))) (VARIABLE ";" (lambda(args) (format "%s" (nth 0
args)))))) Sp))
+ (parser-generator-set-grammar
+ '((Sp S T) (";" OPEN_ROUND_BRACKET CLOSE_ROUND_BRACKET ECHO IF
OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE)
+ ((Sp S) (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET T
+ CLOSE_CURLY_BRACKET (lambda(args) (format "(when %s %s)" (nth 2
args) (nth 5 args)))))
+ (T (ECHO VARIABLE ";" (lambda(args) (format "(message %s)" (nth 1
args))))
+ (VARIABLE ";" (lambda(args) (format "%s" (nth 0 args)))))) Sp))
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -178,4 +186,4 @@ Each production RHS can optionally contain a
lambda-expression that will be call
(kill-buffer buffer))
```
-[Back to syntax analysis](../../Syntax-Analysis.md)
+[Back to syntax analysis](../Syntax-Analysis.md)
diff --git a/test/parser-generator-lex-analyzer-test.el
b/test/parser-generator-lex-analyzer-test.el
index ea93b8f..4e2134f 100644
--- a/test/parser-generator-lex-analyzer-test.el
+++ b/test/parser-generator-lex-analyzer-test.el
@@ -113,6 +113,7 @@
(message "Passed failing lex analysis 2")
+ (parser-generator-lex-analyzer--reset)
(setq parser-generator--look-ahead-number 1)
(should
- [elpa] externals/parser-generator 07320b9 249/434: Updated test-case k=2, (continued)
- [elpa] externals/parser-generator 07320b9 249/434: Updated test-case k=2, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator d49f74f 244/434: Added failing test for action-tables LRk parser k=2, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator fe05328 250/434: Passed unit tests for LRk parser k=2, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator d1f4682 248/434: Added a function that converts a FIRST-item to a look-ahead item, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 6845262 243/434: Passed GOTO-tables k=2, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 3a0d4cd 255/434: Added translation test for LRk=2, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 03ca2e9 247/434: Look-ahead generation now supports EOF symbol, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 3563720 266/434: Updated docs about lexical analysis, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 412fd2a 273/434: Passing generation of GOTO-table for LR(0) grammar, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator bc23604 274/434: Fixed look-ahead, first and e-free-first with k = 0, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 1879cb0 271/434: Merge branch 'master' of git.cvj.se:/home/git/emacs-parser-generator,
ELPA Syncer <=
- [elpa] externals/parser-generator 16bb2d8 262/434: Added translation example LRk parser, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator d173c1e 281/434: More work on exporting parser as code function, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator bd887ff 277/434: LR(0) Parser passing translation test, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 852f969 283/434: Merge branch 'master' of git.cvj.se:/home/git/emacs-parser-generator, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 0d6d46d 294/434: Debugging incremental issue with exported translator, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator c4c68b2 302/434: Added progress-indicator to goto-table generation, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 586789d 308/434: Added cache to first calculation, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 236a124 322/434: More work on resolving conflicts, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 04c360b 318/434: LR-items set validation now supports symbols with attributes, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator ea02245 325/434: Improved debug output, ELPA Syncer, 2021/11/29