Re: Valgrind and Bison memory leaks

[Joel E. Denny]

On Thu, 9 Nov 2006, Joel E. Denny wrote:

> I switched to Flex 2.5.33, and now I see the same results as you.  (I 
> didn't bother testing my original patch this time.)  I'll investigate 
> solutions later.

I committed the following patch, which appears to fix it.

Please review my change to translate_action in scan-code.l.  I removed a 
couple of lines that don't appear to do any harm but also don't appear to 
serve any purpose.

Index: ChangeLog
===================================================================
RCS file: /sources/bison/bison/ChangeLog,v
retrieving revision 1.1602
diff -p -u -r1.1602 ChangeLog
--- ChangeLog   9 Nov 2006 18:17:05 -0000       1.1602
+++ ChangeLog   10 Nov 2006 05:17:37 -0000
@@ -1,5 +1,19 @@
 2006-11-10  Joel E. Denny  <address@hidden>
 
+       Fix memory leaks in scanners generated by at least Flex 2.5.9 and
+       later.  Reported by Paul Eggert in
+       <http://lists.gnu.org/archive/html/bison-patches/2006-11/msg00014.html>.
+       * src/flex-scanner.h (yylex_destroy): Define for Flex before 2.5.9.
+       * src/scan-code.l (translate_action): Don't bother invoking
+       yy_delete_buffer (YY_CURRENT_BUFFER) before creating the first buffer.
+       (code_scanner_free): Instead of invoking
+       yy_delete_buffer (YY_CURRENT_BUFFER) directly, invoke yylex_destroy,
+       which frees more.
+       * src/scan-gram.l (gram_scanner_free): Likewise.
+       * src/scan-skel.l (scan_skel): Likewise.
+
+2006-11-09  Joel E. Denny  <address@hidden>
+
        * src/files.c (tr): Change return type to void.
        * src/muscle_tab.c (muscle_insert): Free storage in case muscle_grow
        has been called previously for the same key.
Index: src/flex-scanner.h
===================================================================
RCS file: /sources/bison/bison/src/flex-scanner.h,v
retrieving revision 1.5
diff -p -u -r1.5 flex-scanner.h
--- src/flex-scanner.h  10 Aug 2006 04:53:04 -0000      1.5
+++ src/flex-scanner.h  10 Nov 2006 05:17:37 -0000
@@ -50,6 +50,19 @@ int   FLEX_PREFIX (lex_destroy) (void);
 # define yytext  FLEX_PREFIX (text)
 #endif
 
+/* Non-reentrant scanners generated by Flex 2.5.9 and later (and some earlier
+   versions according to the Flex manual) leak memory if yylex_destroy is not
+   invoked.  However, yylex_destroy is not defined before Flex 2.5.9, so give
+   an implementation here that at least appears to work with Flex 2.5.4.  */
+#if !defined(YY_FLEX_MAJOR_VERSION) || YY_FLEX_MAJOR_VERSION < 2 \
+    || (YY_FLEX_MAJOR_VERSION == 2 \
+        && (!defined(YY_FLEX_MINOR_VERSION) || YY_FLEX_MINOR_VERSION < 5 \
+            || (YY_FLEX_MINOR_VERSION == 5 \
+                && (!defined(YY_FLEX_SUBMINOR_VERSION) \
+                    || YY_FLEX_SUBMINOR_VERSION < 9))))
+# define yylex_destroy() yy_delete_buffer (YY_CURRENT_BUFFER)
+#endif
+
 /* OBSTACK_FOR_STRING -- Used to store all the characters that we need to
    keep (to construct ID, STRINGS etc.).  Use the following macros to
    use it.
Index: src/scan-code.l
===================================================================
RCS file: /sources/bison/bison/src/scan-code.l,v
retrieving revision 1.14
diff -p -u -r1.14 scan-code.l
--- src/scan-code.l     8 Nov 2006 20:28:57 -0000       1.14
+++ src/scan-code.l     10 Nov 2006 05:17:37 -0000
@@ -380,8 +380,6 @@ translate_action (int sc_context, symbol
   if (!initialized)
     {
       obstack_init (&obstack_for_string);
-      /* The initial buffer, never used. */
-      yy_delete_buffer (YY_CURRENT_BUFFER);
       yy_flex_debug = 0;
       initialized = true;
     }
@@ -422,5 +420,5 @@ code_scanner_free (void)
 {
   obstack_free (&obstack_for_string, 0);
   /* Reclaim Flex's buffers.  */
-  yy_delete_buffer (YY_CURRENT_BUFFER);
+  yylex_destroy ();
 }
Index: src/scan-gram.l
===================================================================
RCS file: /sources/bison/bison/src/scan-gram.l,v
retrieving revision 1.107
diff -p -u -r1.107 scan-gram.l
--- src/scan-gram.l     8 Nov 2006 20:28:57 -0000       1.107
+++ src/scan-gram.l     10 Nov 2006 05:17:37 -0000
@@ -800,5 +800,5 @@ gram_scanner_free (void)
 {
   obstack_free (&obstack_for_string, 0);
   /* Reclaim Flex's buffers.  */
-  yy_delete_buffer (YY_CURRENT_BUFFER);
+  yylex_destroy ();
 }
Index: src/scan-skel.l
===================================================================
RCS file: /sources/bison/bison/src/scan-skel.l,v
retrieving revision 1.41
diff -p -u -r1.41 scan-skel.l
--- src/scan-skel.l     10 Aug 2006 04:53:04 -0000      1.41
+++ src/scan-skel.l     10 Nov 2006 05:17:37 -0000
@@ -121,5 +121,5 @@ scan_skel (FILE *in)
   skel__flex_debug = trace_flag & trace_skeleton;
   skel_lex ();
   /* Reclaim Flex's buffers.  */
-  yy_delete_buffer (YY_CURRENT_BUFFER);
+  yylex_destroy ();
 }