forward-comment and syntax-ppss (was: Preview: portable dumper)

[Stefan Monnier]

>> Your patch was indeed simpler.  Indeed, it was so simple as actually not
>> to work in general, as well as being ~2 orders of magnitude slower than
>> the current (forward-comment -1).
> You make it sound like it barely ever works and it's always 100
> times slower.  I think this qualifies as "gross misrepresentation".

As a counter-point, I've been using the patch below since this
discussion, without noticing any adverse effects.  I don't doubt that it
will occasionally mis-behave in cc-mode buffers, but I'm not sure it's
an argument against this patch.  After all, there are mostly 3 entities
at play here:
- this patch (i.e. the linkage between forward-commment and syntax-ppss).
- syntax-ppss itself.
- cc-mode.

I think most of the problems that will show up with this patch can be
attributed to the current syntax-ppss implementation (e.g. its inability
to deal reliably with narrowing or with changes of the syntax-table).
Maybe those problems will best be solved not just by changing
syntax-ppss but by providing more info to syntax-ppss (e.g. requiring
major modes who narrow the buffer or change the syntax-table to
indicate to syntax-ppss how to handle it (since there are various
possible choices and syntax-ppss usually can't know which is best or
even which is right)).

So I see this patch as an opportunity to improve syntax-ppss.


        Stefan


diff --git a/src/syntax.c b/src/syntax.c
index d463f7e..b56e808 100644
--- a/src/syntax.c
+++ b/src/syntax.c
@@ -297,6 +297,7 @@ SETUP_SYNTAX_TABLE_FOR_OBJECT (Lisp_Object object,
       gl_state.b_property = 0;
       gl_state.e_property = PTRDIFF_MAX;
       gl_state.offset = 0;
+      return;                /* There are no syntax-table properties here.  */
     }
   else
     {
@@ -607,6 +634,26 @@ find_defun_start (ptrdiff_t pos, ptrdiff_t pos_byte)
       && MODIFF == find_start_modiff)
     return find_start_value;
 
+  if (!NILP (Vcomment_use_syntax_ppss))
+    {
+      EMACS_INT modiffs = CHARS_MODIFF;
+      Lisp_Object ppss = call1 (Qsyntax_ppss, make_number (pos));
+      if (modiffs != CHARS_MODIFF)
+       error ("syntax-ppss modified the buffer!");
+      TEMP_SET_PT_BOTH (opoint, opoint_byte);
+      Lisp_Object boc = Fnth (make_number (8), ppss);
+      if (NUMBERP (boc))
+        {
+          find_start_value = XINT (boc);
+          find_start_value_byte = CHAR_TO_BYTE (find_start_value);
+        }
+      else
+        {
+          find_start_value = pos;
+          find_start_value_byte = pos_byte;
+        }
+      goto found;
+    }
   if (!open_paren_in_column_0_is_defun_start)
     {
       find_start_value = BEGV;
@@ -874,6 +921,7 @@ back_comment (ptrdiff_t from, ptrdiff_t from_byte, 
ptrdiff_t stop,
        case Sopen:
          /* Assume a defun-start point is outside of strings.  */
          if (open_paren_in_column_0_is_defun_start
+              && NILP (Vcomment_use_syntax_ppss)
              && (from == stop
                  || (temp_byte = dec_bytepos (from_byte),
                      FETCH_CHAR (temp_byte) == '\n')))
@@ -3453,10 +3493,7 @@ internalize_parse_state (Lisp_Object external, struct 
lisp_parse_state *state)
   else
     {
       tem = Fcar (external);
-      if (!NILP (tem))
-       state->depth = XINT (tem);
-      else
-       state->depth = 0;
+      state->depth = INTEGERP (tem) ? XINT (tem) : 0;
 
       external = Fcdr (external);
       external = Fcdr (external);
@@ -3680,6 +3717,11 @@ void
 syms_of_syntax (void)
 {
   DEFSYM (Qsyntax_table_p, "syntax-table-p");
+  DEFSYM (Qsyntax_ppss, "syntax-ppss");
+  DEFVAR_LISP ("comment-use-syntax-ppss",
+              Vcomment_use_syntax_ppss,
+              doc: /* Non-nil means `forward-comment' can use `syntax-ppss' 
internally.  */);
+  Vcomment_use_syntax_ppss = Qt;
 
   staticpro (&Vsyntax_code_object);