bison-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: My plans for Bison: reentrant and pure


From: Akim Demaille
Subject: Re: My plans for Bison: reentrant and pure
Date: Sun, 17 Feb 2019 11:35:20 +0100


> Le 16 févr. 2019 à 13:54, Eric S. Raymond <address@hidden> a écrit :
> 
> Akim Demaille <address@hidden>:
>> To demonstrate reentrancy, this calculator invokes another parser
>> on parens (whose content is kept a string).  So (((1)+(2))*((3)+(4)))
>> uses 8 parsers, with a depth of 4.
>> 
>> Here are the files (easier than the diff).  Comments most welcome.
>> Please let me know if it addresses all your concerns.
> 
> From the files, looks like it does. I may have more comments when I read it
> in context.

Here is what I installed.  Cheers!

commit 6289d673a0e8e82cee1aff35a45e6ee5e23c6952
Author: Akim Demaille <address@hidden>
Date:   Sat Feb 16 13:13:30 2019 +0100

    examples: add an example with a reentrant parser in Flex+Bison
    
    Suggested by Eric S. Raymond.
    https://lists.gnu.org/archive/html/bison-patches/2019-02/msg00066.html
    
    * examples/c/reentrant-calc/Makefile, examples/c/reentrant-calc/README.md,
    * examples/c/reentrant-calc/parse.y, examples/c/reentrant-calc/scan.l
    * examples/c/reentrant-calc/lexcalc.test,
    * examples/c/reentrant-calc/local.mk:
    New.

diff --git a/THANKS b/THANKS
index 597ca3f7..6ddfe694 100644
--- a/THANKS
+++ b/THANKS
@@ -56,6 +56,7 @@ Didier Godefroy           address@hidden
 Efi Fogel                 address@hidden
 Enrico Scholz             address@hidden
 Eric Blake                address@hidden
+Eric S. Raymond           address@hidden
 Étienne Renault           address@hidden
 Evgeny Stambulchik        address@hidden
 Fabrice Bauzac            address@hidden
diff --git a/examples/c/README.md b/examples/c/README.md
index 6cdbcfe3..46f45634 100644
--- a/examples/c/README.md
+++ b/examples/c/README.md
@@ -2,7 +2,7 @@
 
 This directory contains simple examples of Bison grammar files in C.
 
-Most of them come from the documentation, which should be installed together
+Some of them come from the documentation, which should be installed together
 with Bison.  The URLs are provided for convenience.
 
 ## rpcalc - Reverse Polish Notation Calculator
@@ -16,7 +16,7 @@ 
https://www.gnu.org/software/bison/manual/html_node/RPN-Calc.html
 ## calc - Simple Calculator
 This example is slightly more complex than rpcalc: it features infix
 operators (`1 + 2`, instead of `1 2 +` in rpcalc), but it does so using a
-unambiguous grammar of the arithmetics instead of using precedence
+unambiguous grammar of the arithmetic instead of using precedence
 directives (%left, etc.).
 
 ## mfcalc - Multi-Function Calculator
@@ -30,6 +30,13 @@ 
https://www.gnu.org/software/bison/manual/html_node/Multi_002dfunction-Calc.html
 ## lexcalc - calculator with Flex and Bison
 The calculator, redux.  This time using a scanner generated by Flex.
 
+## reccalc - recursive calculator with Flex and Bison
+The example builds on top of the previous one to provide a reentrant parser.
+Such parsers can be called concurrently in different threads, or even
+recursively.  To demonstrate this feature, expressions in parentheses are
+tokenized as strings, and then recursively parsed from the parser.  So
+`(((1)+(2))*((3)+(4)))` uses eight parsers, with a depth of four.
+
 
 <!---
 
@@ -47,5 +54,6 @@ Invariant Sections, with no Front-Cover Texts, and with no 
Back-Cover
 Texts.  A copy of the license is included in the "GNU Free
 Documentation License" file as part of this distribution.
 
-# LocalWords:  mfcalc calc parsers yy
+# LocalWords:  mfcalc calc parsers yy rpcalc lexcalc redux reccalc ispell
+# LocalWords:  reentrant tokenized american postfix
 --->
diff --git a/examples/c/local.mk b/examples/c/local.mk
index dce4b1f7..11071f1a 100644
--- a/examples/c/local.mk
+++ b/examples/c/local.mk
@@ -19,4 +19,5 @@ dist_c_DATA = %D%/README.md
 include %D%/calc/local.mk
 include %D%/lexcalc/local.mk
 include %D%/mfcalc/local.mk
+include %D%/reccalc/local.mk
 include %D%/rpcalc/local.mk
diff --git a/examples/c/reccalc/Makefile b/examples/c/reccalc/Makefile
new file mode 100644
index 00000000..84fbd3ee
--- /dev/null
+++ b/examples/c/reccalc/Makefile
@@ -0,0 +1,35 @@
+# This Makefile is designed to be simple and readable.  It does not
+# aim at portability.  It requires GNU Make.
+
+BASE = reccalc
+BISON = bison
+FLEX = flex
+XSLTPROC = xsltproc
+
+all: $(BASE)
+
+%.c %.h %.xml %.gv: %.y
+       $(BISON) $(BISONFLAGS) --defines --xml --graph=$*.gv -o $*.c $<
+
+%.c %.h: %.l
+       $(FLEX) $(FLEXFLAGS) -o$*.c --header-file=$*.h $<
+
+scan.o: parse.h
+parse.o: scan.h
+$(BASE): parse.o scan.o
+       $(CC) $(CFLAGS) -o $@ $^
+
+run: $(BASE)
+       @echo "Type arithmetic expressions.  Quit with ctrl-d."
+       ./$<
+
+html: parse.html
+%.html: %.xml
+       $(XSLTPROC) $(XSLTPROCFLAGS) -o $@ $$($(BISON) 
--print-datadir)/xslt/xml2xhtml.xsl $<
+
+CLEANFILES =                                           \
+  $(BASE) *.o                                          \
+  parse.[ch] parse.output parse.xml parse.html parse.gv        \
+  scan.[ch]
+clean:
+       rm -f $(CLEANFILES)
diff --git a/examples/c/reccalc/README.md b/examples/c/reccalc/README.md
new file mode 100644
index 00000000..01ba6d32
--- /dev/null
+++ b/examples/c/reccalc/README.md
@@ -0,0 +1,45 @@
+# reccalc - recursive calculator with Flex and Bison
+
+In this example the generated parser is pure and reentrant: it can be used
+concurrently in different threads, or recursively.  As a proof of this
+reentrancy, expressions in parenthesis are tokenized as strings, and then
+recursively parsed from the parser:
+
+```
+exp: STR
+  {
+    result r = parse_string ($1);
+    free ($1);
+    if (r.nerrs)
+      {
+        res->nerrs += r.nerrs;
+        YYERROR;
+      }
+    else
+      $$ = r.value;
+  }
+```
+
+<!---
+Local Variables:
+fill-column: 76
+ispell-dictionary: "american"
+End:
+
+Copyright (C) 2018-2019 Free Software Foundation, Inc.
+
+This file is part of Bison, the GNU Compiler Compiler.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+--->
diff --git a/examples/c/reccalc/local.mk b/examples/c/reccalc/local.mk
new file mode 100644
index 00000000..bf29aa33
--- /dev/null
+++ b/examples/c/reccalc/local.mk
@@ -0,0 +1,53 @@
+## Copyright (C) 2019 Free Software Foundation, Inc.
+##
+## This program is free software: you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation, either version 3 of the License, or
+## (at your option) any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+reccalcdir = $(docdir)/%D%
+
+## ------ ##
+## Calc.  ##
+## ------ ##
+
+check_PROGRAMS += %D%/reccalc
+TESTS += %D%/reccalc.test
+EXTRA_DIST += %D%/reccalc.test %D%/scan.l
+%C%_reccalc_SOURCES = %D%/parse.y %D%/parse.h
+nodist_%C%_reccalc_SOURCES = %D%/scan.h %D%/scan.c
+BUILT_SOURCES += $(nodist_%C%_reccalc_SOURCES)
+%D%/parse.c: $(dependencies)
+
+# Tell Make that parse.o depends on scan.h, so that scan.h is built
+# before parse.o.  Obfuscate the name of the target, otherwise
+# Automake removes its recipe for parse.o and leaves only our
+# additional dependency.
+DASH = -
+%D%/reccalc$(DASH)parse.o: %D%/scan.h
+
+%D%/scan.c %D%/scan.h: %D%/scan.stamp
+       @test -f $@ || rm -f %D%/scan.stamp
+       @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) %D%/scan.stamp
+
+%D%/scan.stamp: %D%/scan.l
+       $(AM_V_LEX)rm -f $@ address@hidden
+       $(AM_V_at)$(MKDIR_P) %D%
+       $(AM_V_at)touch address@hidden
+       $(AM_V_at) $(LEX) -o %D%/scan.c --header-file=%D%/scan.h $<
+       $(AM_V_at)mv address@hidden $@
+
+# Don't use gnulib's system headers.
+%C%_reccalc_CPPFLAGS = -I$(top_srcdir)/%D% -I$(top_builddir)/%D%
+
+dist_reccalc_DATA = %D%/parse.y %D%/scan.l %D%/Makefile %D%/README.md
+CLEANFILES += %D%/reccalc %D%/*.o %D%/parse.[ch] %D%/scan.[ch] %D%/*.stamp
+CLEANDIRS += %D%/*.dSYM
diff --git a/examples/c/reccalc/parse.y b/examples/c/reccalc/parse.y
new file mode 100644
index 00000000..6d645294
--- /dev/null
+++ b/examples/c/reccalc/parse.y
@@ -0,0 +1,183 @@
+// Prologue (directives).
+%expect 0
+
+// Emitted in the header file, before the definition of YYSTYPE.
+%code requires
+{
+  typedef void* yyscan_t;
+  typedef struct
+  {
+    // Whether to print the intermediate results.
+    int verbose;
+    // Value of the last computation.
+    int value;
+    // Number of errors.
+    int nerrs;
+  } result;
+}
+
+// Emitted in the header file, after the definition of YYSTYPE.
+%code provides
+{
+  // Tell Flex the expected prototype of yylex.
+  // The scanner argument must be named yyscanner.
+#define YY_DECL                                                         \
+  enum yytokentype yylex (YYSTYPE* yylval, yyscan_t yyscanner, result *res)
+  YY_DECL;
+
+  void yyerror (yyscan_t scanner, result *res, const char *msg, ...);
+}
+
+// Emitted on top of the implementation file.
+%code top
+{
+#include <stdarg.h> // va_list.
+#include <stdio.h>  // printf.
+#include <stdlib.h> // getenv.
+}
+
+%code
+{
+  result parse_string (const char* cp);
+  result parse (void);
+}
+
+%define api.pure full
+%define api.token.prefix {TOK_}
+%define api.value.type union
+%define parse.error verbose
+%define parse.trace
+%verbose
+
+ // Scanner and error count are exchanged between main, yyparse and yylex.
+%param {yyscan_t scanner}{result *res}
+
+%token
+  PLUS   "+"
+  MINUS  "-"
+  STAR   "*"
+  SLASH  "/"
+  EOL    "end-of-line"
+  EOF 0  "end-of-file"
+;
+
+%token <int> NUM "number"
+%type <int> exp
+%printer { fprintf (yyo, "%d", $$); } <int>
+
+%token <char*> STR "string"
+%printer { fprintf (yyo, "\"%s\"", $$); } <char*>
+%destructor { free ($$); } <char*>
+
+// Precedence (from lowest to highest) and associativity.
+%left "+" "-"
+%left "*" "/"
+%precedence UNARY
+
+%%
+// Rules.
+input:
+  line
+| input line
+;
+
+line:
+  exp eol
+    {
+      res->value = $exp;
+      if (res->verbose)
+        printf ("%d\n", $exp);
+    }
+| error eol
+    {
+      yyerrok;
+    }
+;
+
+eol:
+  EOF
+| EOL
+;
+
+exp:
+  NUM           { $$ = $1; }
+| exp "+" exp   { $$ = $1 + $3; }
+| exp "-" exp   { $$ = $1 - $3; }
+| exp "*" exp   { $$ = $1 * $3; }
+| exp "/" exp
+  {
+    if ($3 == 0)
+      {
+        yyerror (scanner, res, "invalid division by zero");
+        YYERROR;
+      }
+    else
+      $$ = $1 / $3;
+  }
+| "+" exp %prec UNARY  { $$ = + $2; }
+| "-" exp %prec UNARY  { $$ = - $2; }
+| STR
+  {
+    result r = parse_string ($1);
+    free ($1);
+    if (r.nerrs)
+      {
+        res->nerrs += r.nerrs;
+        YYERROR;
+      }
+    else
+      $$ = r.value;
+  }
+;
+
+%%
+// Epilogue (C code).
+
+#include "scan.h"
+
+result
+parse (void)
+{
+  yyscan_t scanner;
+  yylex_init (&scanner);
+  result res = {1, 0, 0};
+  yyparse (scanner, &res);
+  yylex_destroy (scanner);
+  return res;
+}
+
+result
+parse_string (const char *str)
+{
+  yyscan_t scanner;
+  yylex_init (&scanner);
+  YY_BUFFER_STATE buf = yy_scan_string (str ? str : "", scanner);
+  result res = {0, 0, 0};
+  yyparse (scanner, &res);
+  yy_delete_buffer (buf, scanner);
+  yylex_destroy (scanner);
+  return res;
+}
+
+void
+yyerror (yyscan_t scanner, result *res,
+         const char *msg, ...)
+{
+  (void) scanner;
+  va_list args;
+  va_start (args, msg);
+  vfprintf (stderr, msg, args);
+  va_end (args);
+  fputc ('\n', stderr);
+  res->nerrs += 1;
+}
+
+int
+main (void)
+{
+  // Possibly enable parser runtime debugging.
+  yydebug = !!getenv ("YYDEBUG");
+  result res = parse ();
+  // Exit on failure if there were errors.
+  return !!res.nerrs;
+}
diff --git a/examples/c/reccalc/reccalc.test b/examples/c/reccalc/reccalc.test
new file mode 100644
index 00000000..280fb182
--- /dev/null
+++ b/examples/c/reccalc/reccalc.test
@@ -0,0 +1,57 @@
+#! /bin/sh
+
+# Copyright (C) 2018-2019 Free Software Foundation, Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+cat >input <<EOF
+1+2*3
+EOF
+run 0 '7'
+
+cat >input <<EOF
+(1+2) * 3
+EOF
+run 0 '9'
+run -noerr 0 '9' -p
+
+cat >input <<EOF
+(((1)+(2))*((3)+(4)))
+EOF
+run 0 '21'
+
+# Some really deep computation.
+# for 4, gives 4 + (3 + (2 + (1 + (- (4 * (4 + 1)) / 2)))).
+n=100
+for i in $(seq 0 $n)
+do
+    if [ "$i" -eq 0 ]; then
+        input="- ($n * ($n + 1)) / 2"
+    else
+        input="$i + ($input)"
+    fi
+done
+echo "$input" > input
+run 0 '0'
+         
+cat >input <<EOF
+() + ()
+EOF
+run 1 'err: syntax error, unexpected end-of-file, expecting + or - or number 
or string'
+
+cat >input <<EOF
+1 + $
+EOF
+run 1 'err: syntax error, invalid character: $
+err: syntax error, unexpected end-of-line, expecting + or - or number or 
string'
diff --git a/examples/c/reccalc/scan.l b/examples/c/reccalc/scan.l
new file mode 100644
index 00000000..7cf0ceef
--- /dev/null
+++ b/examples/c/reccalc/scan.l
@@ -0,0 +1,85 @@
+/* Prologue (directives).   -*- C++ -*- */
+
+/* Disable Flex features we don't need, to avoid warnings. */
+%option nodefault noinput nounput noyywrap
+
+%option reentrant
+
+%{
+#include <assert.h>
+#include <limits.h> /* INT_MIN */
+#include <stdlib.h> /* strtol */
+
+#include "parse.h"
+%}
+
+%x SC_STRING
+
+%%
+%{
+  int nesting = 0;
+  char *str = NULL;
+  int size = 0;
+  int capacity = 0;
+#define STR_APPEND()                                    \
+  do {                                                  \
+    if (capacity < size + yyleng + 1)                   \
+      {                                                 \
+        do                                              \
+          capacity = capacity ? 2 * capacity : 128;     \
+        while (capacity < size + yyleng + 1);           \
+        str = realloc (str, capacity);                  \
+      }                                                 \
+    strncpy (str + size, yytext, yyleng);               \
+    size += yyleng;                                     \
+    assert (size < capacity);                           \
+  } while (0)
+%}
+
+ // Rules.
+
+"+"      return TOK_PLUS;
+"-"      return TOK_MINUS;
+"*"      return TOK_STAR;
+"/"      return TOK_SLASH;
+
+"("      nesting += 1; BEGIN SC_STRING;
+
+ /* Scan an integer.  */
+[0-9]+   {
+  errno = 0;
+  long n = strtol (yytext, NULL, 10);
+  if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
+    yyerror (yyscanner, res, "integer is out of range");
+  yylval->TOK_NUM = (int) n;
+  return TOK_NUM;
+}
+
+ /* Ignore white spaces. */
+[ \t]+   continue;
+
+"\n"     return TOK_EOL;
+
+.        yyerror (yyscanner, res, "syntax error, invalid character: %c", 
yytext[0]);
+
+<SC_STRING>
+{
+  "("+   nesting += yyleng; STR_APPEND ();
+  ")"    {
+    if (!--nesting)
+      {
+        BEGIN INITIAL;
+        if (str)
+          str[size] = 0;
+        yylval->TOK_STR = str;
+        return TOK_STR;
+      }
+    else
+      STR_APPEND ();
+  }
+  [^()]+  STR_APPEND ();
+}
+
+<<EOF>>  return TOK_EOF;
+%%
+/* Epilogue (C code). */
diff --git a/examples/c/reentrant-calc/Makefile 
b/examples/c/reentrant-calc/Makefile
new file mode 100644
index 00000000..b7065db4
--- /dev/null
+++ b/examples/c/reentrant-calc/Makefile
@@ -0,0 +1,35 @@
+# This Makefile is designed to be simple and readable.  It does not
+# aim at portability.  It requires GNU Make.
+
+BASE = lexcalc
+BISON = bison
+FLEX = flex
+XSLTPROC = xsltproc
+
+all: $(BASE)
+
+%.c %.h %.xml %.gv: %.y
+       $(BISON) $(BISONFLAGS) --defines --xml --graph=$*.gv -o $*.c $<
+
+%.c %.h: %.l
+       $(FLEX) $(FLEXFLAGS) -o$*.c --header-file=$*.h $<
+
+scan.o: parse.h
+parse.o: scan.h
+$(BASE): parse.o scan.o
+       $(CC) $(CFLAGS) -o $@ $^
+
+run: $(BASE)
+       @echo "Type arithmetic expressions.  Quit with ctrl-d."
+       ./$<
+
+html: parse.html
+%.html: %.xml
+       $(XSLTPROC) $(XSLTPROCFLAGS) -o $@ $$($(BISON) 
--print-datadir)/xslt/xml2xhtml.xsl $<
+
+CLEANFILES =                                           \
+  $(BASE) *.o                                          \
+  parse.[ch] parse.output parse.xml parse.html parse.gv        \
+  scan.c
+clean:
+       rm -f $(CLEANFILES)
diff --git a/examples/c/reentrant-calc/README.md 
b/examples/c/reentrant-calc/README.md
new file mode 100644
index 00000000..31ef2767
--- /dev/null
+++ b/examples/c/reentrant-calc/README.md
@@ -0,0 +1,28 @@
+# lexcalc - calculator with Flex and Bison
+
+This directory contains lexcalc, the traditional example of using Flex and
+Bison to build a simple calculator.
+
+<!---
+Local Variables:
+fill-column: 76
+ispell-dictionary: "american"
+End:
+
+Copyright (C) 2018-2019 Free Software Foundation, Inc.
+
+This file is part of Bison, the GNU Compiler Compiler.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+--->
diff --git a/examples/c/reentrant-calc/lexcalc.test 
b/examples/c/reentrant-calc/lexcalc.test
new file mode 100644
index 00000000..bd9f3929
--- /dev/null
+++ b/examples/c/reentrant-calc/lexcalc.test
@@ -0,0 +1,33 @@
+#! /bin/sh
+
+# Copyright (C) 2018-2019 Free Software Foundation, Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+cat >input <<EOF
+1+2*3
+EOF
+run 0 7
+
+cat >input <<EOF
+(1+2) * 3
+EOF
+run 0 9
+run -noerr 0 9 -p
+
+cat >input <<EOF
+(((1)+(2))*((3)+(4)))
+EOF
+run 0 21
+
diff --git a/examples/c/reentrant-calc/local.mk 
b/examples/c/reentrant-calc/local.mk
new file mode 100644
index 00000000..bf1d5fb6
--- /dev/null
+++ b/examples/c/reentrant-calc/local.mk
@@ -0,0 +1,37 @@
+## Copyright (C) 2018-2019 Free Software Foundation, Inc.
+##
+## This program is free software: you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation, either version 3 of the License, or
+## (at your option) any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+lexcalcdir = $(docdir)/%D%
+
+## ------ ##
+## Calc.  ##
+## ------ ##
+
+check_PROGRAMS += %D%/lexcalc
+TESTS += %D%/lexcalc.test
+EXTRA_DIST += %D%/lexcalc.test %D%/scan.l
+%C%_lexcalc_SOURCES = %D%/parse.y %D%/parse.h
+nodist_%C%_lexcalc_SOURCES = %D%/scan.c %D%/scan.h
+%D%/lexcalc.c: $(dependencies)
+
+%D%/scan.c: %D%/scan.l
+       $(AM_V_LEX) $(LEX) -o %D%/scan.c --header-file=%D%/scan.h $<
+
+# Don't use gnulib's system headers.
+%C%_lexcalc_CPPFLAGS = -I$(top_srcdir)/%D% -I$(top_builddir)/%D%
+
+dist_lexcalc_DATA = %D%/parse.y %D%/scan.l %D%/Makefile %D%/README.md
+CLEANFILES += %D%/lexcalc %D%/*.o %D%/parse.c %D%/scan.c
+CLEANDIRS += %D%/*.dSYM
diff --git a/examples/c/reentrant-calc/parse.y 
b/examples/c/reentrant-calc/parse.y
new file mode 100644
index 00000000..c04b9119
--- /dev/null
+++ b/examples/c/reentrant-calc/parse.y
@@ -0,0 +1,160 @@
+// Prologue (directives).
+%expect 0
+
+// Emitted in the header file, before the definition of YYSTYPE.
+%code requires
+{
+  typedef void* yyscan_t;
+  typedef struct
+  {
+    // Whether to print the intermediate results.
+    int verbose;
+    // Value of the last computation.
+    int value;
+    // Number of errors.
+    int nerrs;
+  } result;
+}
+
+// Emitted in the header file, after the definition of YYSTYPE.
+%code provides
+{
+  // Tell Flex the expected prototype of yylex.
+  // The scanner argument must be named yyscanner.
+#define YY_DECL                                 \
+  enum yytokentype yylex (YYSTYPE* yylval, yyscan_t yyscanner, result *res)
+  YY_DECL;
+
+  void yyerror (yyscan_t scanner, result *res, const char *msg);
+}
+
+// Emitted on top of the implementation file.
+%code top
+{
+#include <stdio.h>  /* printf. */
+#include <stdlib.h> /* getenv. */
+}
+
+%code
+{
+  result parse_string (const char* cp);
+  result parse (void);
+}
+
+%define api.pure full
+%define api.token.prefix {TOK_}
+%define api.value.type union
+%define parse.error verbose
+%define parse.trace
+ // Scanner and error count are exchanged between main, yyparse and yylex.
+%param {yyscan_t scanner}{result *res}
+
+%token
+  PLUS   "+"
+  MINUS  "-"
+  STAR   "*"
+  SLASH  "/"
+  EOL    "end-of-line"
+  EOF 0  "end-of-file"
+;
+
+%token <int> NUM "number"
+%type <int> exp line
+%printer { fprintf (yyo, "%d", $$); } <int>
+
+%token <char*> STR "string"
+%printer { fprintf (yyo, "\"%s\"", $$); } <char*>
+
+// Precedence (from lowest to highest) and associativity.
+%left "+" "-"
+%left "*" "/"
+
+%%
+// Rules.
+input:
+  %empty
+| input line
+  {
+    res->value = $line;
+    if (res->verbose)
+      printf ("%d\n", $line);
+  }
+;
+
+line:
+  exp EOL   { $$ = $1; }
+| exp       { $$ = $1; }
+| error EOL { $$ = 0; yyerrok; }
+;
+
+exp:
+  exp "+" exp   { $$ = $1 + $3; }
+| exp "-" exp   { $$ = $1 - $3; }
+| exp "*" exp   { $$ = $1 * $3; }
+| exp "/" exp
+  {
+    if ($3 == 0)
+      {
+        yyerror (scanner, res, "invalid division by zero");
+        YYERROR;
+      }
+    else
+      $$ = $1 / $3;
+  }
+| STR
+  {
+    result r = parse_string ($1);
+    free ($1);
+    if (r.nerrs)
+      {
+        res->nerrs += r.nerrs;
+        YYERROR;
+      }
+    else
+      $$ = r.value;
+  }
+| NUM      { $$ = $1; }
+;
+%%
+#include "scan.h"
+
+result
+parse (void)
+{
+  result res = {1, 0, 0};
+  yyscan_t scanner;
+  yylex_init (&scanner);
+  yyparse (scanner, &res);
+  yylex_destroy (scanner);
+  return res;
+}
+
+result
+parse_string (const char *str)
+{
+  result res = {0, 0, 0};
+  yyscan_t scanner;
+  yylex_init (&scanner);
+  YY_BUFFER_STATE buf = yy_scan_string (str, scanner);
+  yyparse (scanner, &res);
+  yy_delete_buffer(buf, scanner);
+  yylex_destroy (scanner);
+  return res;
+}
+
+// Epilogue (C code).
+void yyerror (yyscan_t scanner, result *res, const char *msg)
+{
+  (void) scanner;
+  fprintf (stderr, "%s\n", msg);
+  res->nerrs += 1;
+}
+
+int main (void)
+{
+  // Possibly enable parser runtime debugging.
+  yydebug = !!getenv ("YYDEBUG");
+  result res = parse ();
+  // Exit on failure if there were errors.
+  return !!res.nerrs;
+}
diff --git a/examples/c/reentrant-calc/scan.l b/examples/c/reentrant-calc/scan.l
new file mode 100644
index 00000000..62274733
--- /dev/null
+++ b/examples/c/reentrant-calc/scan.l
@@ -0,0 +1,81 @@
+/* Prologue (directives).   -*- C++ -*- */
+
+/* Disable Flex features we don't need, to avoid warnings. */
+%option nodefault noinput nounput noyywrap
+
+%option reentrant
+
+%{
+#include <limits.h> /* INT_MIN */
+#include <stdlib.h> /* strtol */
+
+#include "parse.h"
+%}
+
+%x SC_STRING
+
+%%
+%{
+  int nesting = 0;
+  char *str = NULL;
+  int size = 0;
+  int capacity = 0;
+#define STR_APPEND()                                    \
+  do {                                                  \
+    if (capacity < size + 1)                            \
+      {                                                 \
+        do                                              \
+          capacity = capacity ? 2 * capacity : 128;     \
+        while (capacity < size + 1);                    \
+        str = realloc (str, capacity);                  \
+      }                                                 \
+    strncpy (str + size, yytext, yyleng);               \
+    size += yyleng;                                     \
+  } while (0)
+%}
+
+ // Rules.
+
+"+"      return TOK_PLUS;
+"-"      return TOK_MINUS;
+"*"      return TOK_STAR;
+"/"      return TOK_SLASH;
+
+"("      nesting += 1; BEGIN SC_STRING;
+
+ /* Scan an integer.  */
+[0-9]+   {
+  errno = 0;
+  long n = strtol (yytext, NULL, 10);
+  if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
+    yyerror (yyscanner, res, "integer is out of range");
+  yylval->TOK_NUM = (int) n;
+  return TOK_NUM;
+}
+
+ /* Ignore white spaces. */
+[ \t]+   continue;
+
+"\n"     return TOK_EOL;
+
+.        yyerror (yyscanner, res, "syntax error, invalid character");
+
+<SC_STRING>
+{
+  "("*   nesting += yyleng; STR_APPEND ();
+  ")"    {
+    if (!--nesting)
+      {
+        BEGIN INITIAL;
+        yylval->TOK_STR = str;
+        return TOK_STR;
+      }
+    else
+      STR_APPEND ();
+  }
+  [^()]+  STR_APPEND ();
+}
+
+<<EOF>>  return TOK_EOF;
+%%
+/* Epilogue (C code). */





reply via email to

[Prev in Thread] Current Thread [Next in Thread]