bug#61514: 30.0.50; sadistically long xml line hangs emacs

[Stefan Monnier]

> Looking at the history of that variable, which is in fact a compile-time
> constant, I see that it was initially (May 1995) set to 200000.  A few
> months later (Nov 1995) it was set to 20000, and reduced again (apparently
> because of bug reports) to 8000 and to 4000 (both in Jun 1996).  Two months
> later it was again set to 20000 (Aug 1996), and a year later to 40000 (Dec
> 1997).  It kept that value since then.  As these changes (and this bug
> report) demonstrate, it is not possible to give that variable a "one size
> fits all" value.

Note that the stack is allocated with `SAFE_ALLOCA` and used to be
allocated with just `alloca`.  So the constant was probably reduced
(back in the 90s) in response to reports of segfaults due to
C stack overflows.

Nowadays we should be hopefully(?) safe from such segfaults since
`SAFE_ALLOCA` only uses `alloca` for smallish allocations.

> @@ -731,7 +731,8 @@ xmltok-scan-after-comment-open
>  
>  (defun xmltok-scan-attributes ()
>    (let ((recovering nil)
> -     (atts-needing-normalization nil))
> +     (atts-needing-normalization nil)
> +     (regexp-max-failures 1000))
>      (while (cond ((or (looking-at (xmltok-attribute regexp))
>                     ;; use non-greedy group
>                     (when (looking-at (concat "[^<>\n]+?"

This really needs a comment (at least one referring to this bug report).
I think the idea is that we hope the regexp will need at most one stack
entry per character, so the above means that we're willing to limit the
regexp search to about 1kB of text, which sounds fair given it's
supposed to match just a single XML attribute.

> +  DEFVAR_INT ("regexp-max-failures", Vregexp_max_failures,
> +           doc: /* Maximum number of failures points in a regexp search.  
> */);
> +  Vregexp_max_failures = max_regexp_max_failures;

This name is misleading.  It suggests it's talking about how many times
we fail, whereas the reality is that it's about the number of pending
branches in the search space (which the source code calls "failure
points" because it's info to be used in case the current branch fails
to match).  It could also be described as the number of "pending
continuations" or "stacked failure continuations" or some wording
like that.

But for the var name itself, how 'bout `regexp-max-backtracking-depth`?


        Stefan