Re: question about reserved word file

[Hideki IWAMOTO]

On Sun, 10 Oct 2004 00:36:53 +0900, Shigio Yamaguchi wrote...
> > Gperf source and Perl regular expression are generated from the reserved 
> > word
>  file.
> > Why isn't the Lex definition generated?
> > If the definition of Lex is generated, source code will become simpler.
> 
> How does it become simple?
> Could you show a concrete image?

My interest is only in preprocessing directives.
It will not be useful for other reserved words.

1. In addition to Gperf source, Lex source is also generated by reserved.pl.

  <prefix>_res.in   <prefix>_scan.in
            |           |           
           [ reserved.pl ]
            |           |
            v           v
  <prefix>_res.gpf  <prefix>_scan.l
          |                |           
       [gperf]          [flex]
          |                |
          v                v
  <prefix>_res.h    <prefix>_scan.c

2. reconf.sh will contain the following code.

        if [ -f ${lang}_scan.in ]; then
                perl ./reserved.pl --prefix=$lang --lex ${lang}_res.in \
                    | cat - ${lang}_scan.in > ${lang}_scan.l
                flex -o${lang}_scan.c ${lang}_scan.l
        fi

3. reserved.pl adds the definition of reserved words at the head of 
<prefix>_scan.in. 

  <prefix>_scan.in
  +-------------------------------------
  |%{
  |%}
  |   ....
  |%%
  |   ....
  +-------------------------------------
    |
    v
  <prefix>_scan.l
  +-------------------------------------
  |SHARP_MACROS     assert|define|...
  |%{
  |%}
  |   ....
  |%%
  |   ....
  +-------------------------------------

4. The unknown preprocessing directives can be distinguished by using the 
generated definition.

  asm_scan.in
  +-------------------------------------
  | ....
  |<INITIAL>{
  |  ^[ \t]*\#[ \t]*define         { yy_push_state(PREPROCESSOR_LINE); return 
ASM_DEFINE; }
  |  ^[ \t]*\#[ \t]*undef          { yy_push_state(PREPROCESSOR_LINE); return 
ASM_UNDEF; }
  |  ^[ \t]*\#[ \t]*{SHARP_MACROS} |
  |  ^[ \t]*\#                     { yy_push_state(PREPROCESSOR_LINE); return 
ASM_DIRECTIVE; }
  |  ^[ \t]*\#[ \t]*{WORD}         { yy_push_state(LINE_COMMENT); }
  |  ....
  +-------------------------------------

 When using the lookup function generated by gperf, writing the code for 
removing blank is necessary.

  asm_scan.l
  +-------------------------------------
  | ....
  |#include "asm_res.h"
  | ....
  |<INITIAL>{
  |  ^[ \t]*\#[ \t]*define         { yy_push_state(PREPROCESSOR_LINE); return 
ASM_DEFINE; }
  |  ^[ \t]*\#[ \t]*undef          { yy_push_state(PREPROCESSOR_LINE); return 
ASM_UNDEF; }
  |  ^[ \t]*\#[ \t]*{WORD} { 
  |             int i = 0;
  |             char *p;
  |
  |             p = locatestring(yytext, " ", MATCH_LAST);
  |
  |             if (!p) {
  |                     p = yytext;
  |                     i = yyleng;
  |             } else {
  |                     static char buf[IDENTLEN];
  |
  |                     p++;
  |                     if (*p == '#')
  |                             p++;
  |                     buf[i++] = '#';
  |                     while (*p) {
  |                             buf[i++] = *p++;
  |                             if (i >= sizeof(buf))
  |                                  die("Too long name '%s'.", yytext);
  |                     }
  |                     buf[i] = '\0';
  |                     p = buf;
  |             }
  |             if (i > 0 && reserved_sharp(p, i))
  |                     yy_push_state(PREPROCESSOR_LINE);
  |             else
  |                     yy_push_state(LINE_COMMENT);
  |     }
  |  ^[ \t]*\#                     { yy_push_state(PREPROCESSOR_LINE); return 
ASM_DIRECTIVE; }
  |  ....
  +-------------------------------------
 
  

----
Hideki IWAMOTO  address@hidden