Re: [PATCH] scanner: reintroduce unput for missing end tokens

[Théophile Ranquet]

2012/12/14 Akim Demaille <address@hidden>:
> How do you know the buffer is large enough???

The terminating null byte: we don't need it anymore at this point :)

>>         fwrite (buf, 1, len, out);
>
> I would prefer that you just add the \n afterwards.

Yes, that does sound a lot more reasonable. I would very much have
liked to use 'if (feof (caret_info.source))' as a test, but that is
not true as it would trigger on any last line of a file, even if it
*is* LF terminated.

         fwrite (buf, 1, len, out);
+        if (buf[len - 1] != '\n')
+          fputc ('\n', out);

> And really, I'm
> having second thoughts on using getline here, as a simple while loop
> on getc and putc would perfectly suffice.

Yes, it is true that reading the full line isn't very useful if we
don't cache it, and that is seems to add unwarranted complexity to the
code. However, one could argue that an alternative using fgetc / fputc
wouldn't necessarily be simpler. As you can see, this method (unless I
am doing it wrong) exposes the handling of EOF, which isn't the
sweetest of things:

+static char
+peek (FILE *stream)
+{
+      char c = fgetc (stream);
+      ungetc (c, stream);
+      return c;
 }

[...]

-    char *buf = NULL;
-    size_t size = 0;
-    ssize_t len = getline (&buf, &size, caret_info.source);
-    if (0 < len)
-      {
-        /* The caret of a multiline location ends with the first
line.  */
-        int end = loc.start.line != loc.end.line ? len :
loc.end.column;
+    size_t len = 0;
+    /* Quote the file, indent by a single column.  */
+    {
+      char c;
+      /* Dirty hack to avoid indenting if there is nothing to output.
 */
+      if (peek (caret_info.source) == EOF)
+        return;
+      fputc (' ', out);
+      do
+        {
+          ++len;
+          c = fgetc (caret_info.source);
+          /* If the line is not terminated by a line feed, add one.
*/
+          if (c == EOF)
+            fputc ('\n', out);
+          else
+            fputc (c, out);
+        } while (c != EOF && c != '\n');
+    }

-        /* Quote the file, indent by a single column.  */
-        fputc (' ', out);
-        fwrite (buf, 1, len, out);
-        if (buf[len - 1] != '\n')
-          fputc ('\n', out);

+    /* The caret of a multiline location ends with the first line.
*/
+    len = loc.start.line != loc.end.line ? len : loc.end.column;

I am not quite sure how to avoid using peek() without introducing an
additional boolean, or an obstack, all of those solutions being quite
different from the "simple while loop on getc and putc" that you were
recommending.