bug#24358: 25.1.50; re-search-forward errors with "Variable binding depth exceeds max-specpdl-size"

[npostavs]

# this cloned bug doesn't have a patch yet
tags 24358 - patch
quit

npostavs@users.sourceforge.net writes:

> (I'm also on GNU/Linux, Arch) I get the same max-specpdl-size error with
> 25.1.50, with 24.5 (and below) I get (error "Stack overflow in regexp
> matcher")

The problem is that re_max_failures is set to 40001 (instead of the
original 40000) in main()[1], which is a problem because of the
GROW_FAIL_STACK uses (re_max_failures * TYPICAL_FAILURE_SIZE) as a cap
on the amount to allocate, but ((fail_stack).size * sizeof
(fail_stack_elt_t)) to calculate current allocation.

Since TYPICAL_FAILURE_SIZE = 20 and sizeof (fail_stack_elt_t) = 8, and
it seems that (fail_stack).size grows in increments of 3, when
(fail_stack).avail is 99999 and (fail_stack).size reaches 100002:

(fail_stack).size * sizeof (fail_stack_elt_t) => 800016
  re_max_failures * TYPICAL_FAILURE_SIZE      => 800020

ENSURE_FAIL_STACK(3) then loops indefinitely reallocating a stack of
size 800020 again and again until the record_xmalloc fails to
grow_specdl() (thus the "max-specpdl-size" error).

----------

So we we might want to fix the re_max_failures setting in main, but it
doesn't quite make sense to me that GROW_FAIL_STACK relies on
re_max_failures being a multiple of (sizeof (fail_stack_elt_t)).  At the
definition of TYPICAL_FAILURE_SIZE we have

/* Estimate the size of data pushed by a typical failure stack entry.
   An estimate is all we need, because all we use this for
   is to choose a limit for how big to make the failure stack.  */
/* BEWARE, the value `20' is hard-coded in emacs.c:main().  */
#define TYPICAL_FAILURE_SIZE 20

Why do we use an "estimate" here?  What's wrong with just using
(re_max_failures * sizeof (fail_stack_elt_t)) as the limit?  Or should
the limit actually be (re_max_failures * TYPICAL_FAILURE_SIZE * sizeof
(fail_stack_elt_t))?

-----------

827           long lim = rlim.rlim_cur;
(gdb) p rlim
$1 = {
  rlim_cur = 8388608, 
  rlim_max = 18446744073709551615
}
(gdb) next
833           int ratio = 20 * sizeof (char *);
(gdb) 
834           ratio += ratio / 3;
(gdb) 
837           int extra = 200000;
(gdb) p ratio
$2 = 213
[...]
(gdb) display ((newlim - extra) / ratio)
1: ((newlim - extra) / ratio) = 40000
(gdb) next
856               newlim += pagesize - 1;
1: ((newlim - extra) / ratio) = 40000
(gdb) 
857               if (0 <= rlim.rlim_max && rlim.rlim_max < newlim)
1: ((newlim - extra) / ratio) = 40019
(gdb) 
859               newlim -= newlim % pagesize;
1: ((newlim - extra) / ratio) = 40019
(gdb) 
861               if (pagesize <= newlim - lim)
1: ((newlim - extra) / ratio) = 40001
(gdb) undisplay 1
(gdb) next
863                   rlim.rlim_cur = newlim;
(gdb) 
864                   if (setrlimit (RLIMIT_STACK, &rlim) == 0)
(gdb) 
865                     lim = newlim;
(gdb) 
870           re_max_failures = lim < extra ? 0 : min (lim - extra, SIZE_MAX) / 
ratio;
(gdb) 
875       stack_bottom = &stack_bottom_variable;
(gdb) p re_max_failures
$3 = 40001

-----------

[1]: This was the case since 9d356f62 2016-05-27 "Robustify stack-size 
calculation"