strdup (yytext)

[Akim Demaille]

This piece of FAQ was really needed.  I'm not convinced by my style
though, so any initiative will be most appreciated :)

2003-03-29  Akim Demaille  <address@hidden>

        * doc/bison.texinfo (Strings are Destroyed): New.

Index: doc/bison.texinfo
===================================================================
RCS file: /cvsroot/bison/bison/doc/bison.texinfo,v
retrieving revision 1.102
diff -u -u -r1.102 bison.texinfo
--- doc/bison.texinfo 2 Mar 2003 15:11:43 -0000 1.102
+++ doc/bison.texinfo 29 Mar 2003 11:17:11 -0000
@@ -284,6 +284,7 @@
 Frequently Asked Questions
 
 * Parser Stack Overflow::      Breaking the Stack Limits
+* Strings are Destroyed::      @code{yylval} Loses Track of Strings
 
 Copying This Manual
 
@@ -6352,6 +6353,7 @@
 
 @menu
 * Parser Stack Overflow::      Breaking the Stack Limits
+* Strings are Destroyed::      @code{yylval} Loses Track of Strings
 @end menu
 
 @node Parser Stack Overflow
@@ -6364,6 +6366,65 @@
 
 This question is already addressed elsewhere, @xref{Recursion,
 ,Recursive Rules}.
+
address@hidden Strings are Destroyed
address@hidden Strings are Destroyed
+
address@hidden
+My parser seems to destroy old strings, or maybe it losses track of
+them.  Instead of reporting @samp{"foo", "bar"}, it reports
address@hidden"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
address@hidden display
+
+This error is probably the single most frequent ``bug report'' sent to
+Bison lists, but is only concerned with a misunderstanding of the role
+of scanner.  Consider the following Lex code:
+
address@hidden
+%{
+#include <stdio.h>
+char *yylval = NULL;
+%}
+%%
+.*    yylval = yytext; return 1;
+\n    /* IGNORE */
+%%
+int
+main ()
+{
+  /* Similar to using $1, $2 in a Bison action. */
+  char *fst = (yylex (), yylval);
+  char *snd = (yylex (), yylval);
+  printf ("\"%s\", \"%s\"\n", fst, snd);
+  return 0;
+}
address@hidden verbatim
+
+If you compile and run this code, you get:
+
address@hidden
+$ @kbd{flex -osplit-lines.c split-lines.l}
+$ @kbd{gcc  -osplit-lines   split-lines.c -ll}
+$ @kbd{printf 'one\ntwo\n' | ./split-lines}
+"one
+two", "two"
address@hidden example
+
address@hidden
+this is because @code{yytext} is a buffer provided for @emph{reading}
+in the action, but if you want to keep it, you have to duplicate it
+(e.g., using @code{strdup}).  Note that the output may depend on how
+your implementation of Lex handles @code{yytext}.  For instance, when
+given the Lex compatibility option @option{-l} (which triggers the
+option @samp{%array}) Flex generates a different behavior:
+
address@hidden
+$ @kbd{flex -l -osplit-lines.c split-lines.l}
+$ @kbd{gcc     -osplit-lines   split-lines.c -ll}
+$ @kbd{printf 'one\ntwo\n' | ./split-lines}
+"two", "two"
address@hidden example
+
 
 @c ================================================= Table of Symbols