bug#40661: Crash in regex search during redisplay

[Eli Zaretskii]

> From: Richard Copley <rcopley@gmail.com>
> Date: Thu, 16 Apr 2020 20:35:19 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, 40661@debbugs.gnu.org
> 
> On Thu, 16 Apr 2020 at 18:24, Daniel Colascione <dancol@dancol.org> wrote:
> >
> > On 4/16/20 9:56 AM, Richard Copley wrote:
> > > On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol@dancol.org> wrote:
> > >>
> > >> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote:
> > >>>> Date: Thu, 16 Apr 2020 18:36:36 +0300
> > >>>> From: Eli Zaretskii <eliz@gnu.org>
> > >>>> Cc: 40661@debbugs.gnu.org
> > >>>>
> > >>>> Looks like GC sometimes kicks in while we are inside re_search_2
> > >>>
> > >>> Or not.  I cannot get a breakpoint inside GC to fire while we are in
> > >>> search_buffer_re, so maybe my hypothesis was wrong.  Although the
> > >>> symptoms are all there: when the segfault hits, the pointers passed to
> > >>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
> > >>> they are supposed to be computed, are valid (and different).  And the
> > >>> patch does seem to avoid the segfaults.  But maybe it's just a
> > >>> coincidence or a side effect...
> > >>
> > >> Try using rr and see where those pointers came from
> > >
> > > It seems clear from "str1=str1@entry=0xc607fd", etc., that they come
> > > from the caller, search_buffer_re. The question is, why are they no
> > > longer valid after updating syntax?
> >
> > Right. So let's see what updated the valid pointers and invalidated the
> > invalid ones.
> 
> Right, I see. Anyway, I wasn't able to reproduce the bug under
> GNU/Linux (in order to use rr), or make much progress with GDB on
> Windows.

I finally succeeded to build a sophisticated enough trap to catch the
culprit.  It's GC allright.  Which is not surprising: the commit
pointed out by Richard changed re_match_2_internal to call
UPDATE_SYNTAX_TABLE_FORWARD, which calls Lisp, and thus can trigger
GC.  As seen from the backtrace, GC then calls compact_buffer, which
calls enlarge_buffer_text (to shrink buffer text), and on MS-Windows
-- and this is the w32-specific part -- we return some memory to the
OS and relocate buffer text.

Obviously, we cannot allow GC to run while regex routines do their
work, because they are passed C pointers to buffer text.  The question
is, where to disable GC?  We could do it inside
update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
from many places that evidently have no problems with GC.  So my
suggestion would be to disable GC inside re_match_2_internal instead.

Comments?

Here's the full backtrace I caught:

  mmap_alloc (var=0x6843168, nbytes=2257) at w32heap.c:676
  676           if (*var == NULL)
  #0  mmap_alloc (var=0x6843168, nbytes=2257) at w32heap.c:676
  #1  0x015a7934 in mmap_realloc (var=0x6843168, nbytes=2257) at w32heap.c:784
  #2  0x0124d431 in enlarge_buffer_text (b=0x6842f10, delta=-1840)
      at buffer.c:5049
  #3  0x01262107 in make_gap_smaller (nbytes_removed=1840) at insdel.c:549
  #4  0x0126221c in make_gap (nbytes_added=-1840) at insdel.c:589
  #5  0x01262246 in make_gap_1 (b=0x6842f10, nbytes=-1840) at insdel.c:602
  #6  0x012427e8 in compact_buffer (buffer=0x6842f10) at buffer.c:1672
  #7  0x01314c2e in garbage_collect () at alloc.c:5877
  #8  0x01314b9a in maybe_garbage_collect () at alloc.c:5853
  #9  0x0137696d in maybe_gc () at lisp.h:5065
  #10 0x013848c2 in Ffuncall (nargs=4, args=0x824360) at eval.c:2778
  #11 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865a98),
      vector=XIL(0xa000000006889d58), maxdepth=make_fixnum(7),
      args_template=make_fixnum(257), nargs=1, args=0x824918) at bytecode.c:633
  #12 0x01385af1 in funcall_lambda (fun=XIL(0xa0000000071f2c90), nargs=1,
      arg_vector=0x824910) at eval.c:2989
  #13 0x01384a33 in Ffuncall (nargs=2, args=0x824908) at eval.c:2796
  #14 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865ad8),
      vector=XIL(0xa0000000072005a0), maxdepth=make_fixnum(17),
      args_template=make_fixnum(257), nargs=1, args=0x825058) at bytecode.c:633
  #15 0x01385af1 in funcall_lambda (fun=XIL(0xa0000000071e4b40), nargs=1,
      arg_vector=0x825050) at eval.c:2989
  #16 0x01384a33 in Ffuncall (nargs=2, args=0x825048) at eval.c:2796
  #17 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865a18),
      vector=XIL(0xa000000006828648), maxdepth=make_fixnum(15),
      args_template=make_fixnum(514), nargs=2, args=0x825868) at bytecode.c:633
  #18 0x01385af1 in funcall_lambda (fun=XIL(0xa000000007167768), nargs=2,
      arg_vector=0x825858) at eval.c:2989
  #19 0x01384a33 in Ffuncall (nargs=3, args=0x825850) at eval.c:2796
  #20 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e847e0),
      vector=XIL(0xa000000005e84438), maxdepth=make_fixnum(12),
      args_template=make_fixnum(257), nargs=1, args=0x825e98) at bytecode.c:633
  #21 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e84408), nargs=1,
      arg_vector=0x825e90) at eval.c:2989
  #22 0x01384a33 in Ffuncall (nargs=2, args=0x825e88) at eval.c:2796
  #23 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005ecf1d8),
      vector=XIL(0xa000000005ecf0e0), maxdepth=make_fixnum(8),
      args_template=make_fixnum(257), nargs=1, args=0x826460) at bytecode.c:633
  #24 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005ecf0b0), nargs=1,
      arg_vector=0x826458) at eval.c:2989
  #25 0x01384a33 in Ffuncall (nargs=2, args=0x826450) at eval.c:2796
  #26 0x0137f3e7 in internal_condition_case_n (bfun=0x13847ec <Ffuncall>,
      nargs=2, args=0x826450, handlers=XIL(0x30),
      hfun=0x105a91d <safe_eval_handler>) at eval.c:1435
  #27 0x0105ab54 in safe__call (inhibit_quit=false, nargs=2, func=XIL(0x7fb0),
      ap=0x826514 "") at xdisp.c:2807
  #28 0x0105abcd in safe_call (nargs=2, func=XIL(0x7fb0)) at xdisp.c:2822
  #29 0x0105ac21 in safe_call1 (fn=XIL(0x7fb0), arg=make_fixnum(505))
      at xdisp.c:2833
  #30 0x014128bb in parse_sexp_propertize (charpos=504) at syntax.c:480
  #31 0x01412aa3 in update_syntax_table_forward (charpos=504, init=false,
      object=XIL(0)) at syntax.c:513
  #32 0x012e9cb5 in UPDATE_SYNTAX_TABLE_FORWARD (charpos=504) at syntax.h:185
  #33 0x012e9d39 in UPDATE_SYNTAX_TABLE (charpos=504) at syntax.h:205
  #34 0x012f81f9 in re_match_2_internal (bufp=0x1bb43a0 <searchbufs+4064>,
      string1=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, 
";\
  n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n",
      size1=0,
      string2=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, 
";\
  n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n",
      size2=2051, pos=502, regs=0x1958074 <main_thread+116>, stop=503)
      at regex-emacs.c:4780
  #35 0x012f39f3 in rpl_re_search_2 (bufp=0x1bb43a0 <searchbufs+4064>,
      str1=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, 
";\n",
   '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n", size1=0,
      str2=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, 
";\n",
   '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n",
      size2=2051, startpos=502, range=1, regs=0x1958074 <main_thread+116>,
      stop=503) at regex-emacs.c:3373
  #36 0x012dc6e0 in search_buffer_re (string=XIL(0x8000000006865028), pos=11,
      pos_byte=11, lim=504, lim_byte=504, n=1, trt=XIL(0), inverse_trt=XIL(0),
      posix=false) at search.c:1244
  #37 0x012dd74d in search_buffer (string=XIL(0x8000000006865028), pos=11,
      pos_byte=11, lim=504, lim_byte=504, n=1, RE=1, trt=XIL(0),
      inverse_trt=XIL(0), posix=false) at search.c:1506
  #38 0x012dbefa in search_command (string=XIL(0x8000000006865028),
      bound=make_fixnum(504), noerror=XIL(0x30), count=XIL(0), direction=1,
      RE=1, posix=false) at search.c:1048
  #39 0x012df7bc in Fre_search_forward (regexp=XIL(0x8000000006865028),
      bound=make_fixnum(504), noerror=XIL(0x30), count=XIL(0)) at search.c:2277
  #40 0x01385067 in funcall_subr (subr=0x195ebc0 <Sre_search_forward>,
      numargs=3, args=0x8274d8) at eval.c:2875
  #41 0x013849d9 in Ffuncall (nargs=4, args=0x8274d0) at eval.c:2794
  #42 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eed790),
      vector=XIL(0xa000000005e25f58), maxdepth=make_fixnum(25),
      args_template=make_fixnum(770), nargs=3, args=0x827c38) at bytecode.c:633
  #43 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e25f28), nargs=3,
      arg_vector=0x827c20) at eval.c:2989
  #44 0x01384a33 in Ffuncall (nargs=4, args=0x827c18) at eval.c:2796
  #45 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eee538),
      vector=XIL(0xa000000005e25ac0), maxdepth=make_fixnum(14),
      args_template=make_fixnum(771), nargs=3, args=0x828280) at bytecode.c:633
  #46 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e25a90), nargs=3,
      arg_vector=0x828268) at eval.c:2989
  #47 0x01384a33 in Ffuncall (nargs=4, args=0x828260) at eval.c:2796
  #48 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eee748),
      vector=XIL(0xa000000005e259f0), maxdepth=make_fixnum(7),
      args_template=make_fixnum(770), nargs=2, args=0x8287e8) at bytecode.c:633
  #49 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e259c0), nargs=2,
      arg_vector=0x8287d8) at eval.c:2989
  #50 0x01384a33 in Ffuncall (nargs=3, args=0x8287d0) at eval.c:2796
  #51 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8ab40),
      vector=XIL(0xa000000007255b70), maxdepth=make_fixnum(10),
      args_template=make_fixnum(257), nargs=1, args=0x829060) at bytecode.c:633
  #52 0x01385af1 in funcall_lambda (fun=XIL(0xa000000007255bc0), nargs=1,
      arg_vector=0x829058) at eval.c:2989
  #53 0x01384a33 in Ffuncall (nargs=2, args=0x829050) at eval.c:2796
  #54 0x013837c6 in run_hook_wrapped_funcall (nargs=2, args=0x829050)
      at eval.c:2531
  #55 0x01383ca9 in run_hook_with_args (nargs=2, args=0x829050,
      funcall=0x138377e <run_hook_wrapped_funcall>) at eval.c:2612
  #56 0x01383815 in Frun_hook_wrapped (nargs=2, args=0x829050) at eval.c:2546
  #57 0x01384e99 in funcall_subr (subr=0x1960bc0 <Srun_hook_wrapped>,
      numargs=2, args=0x829050) at eval.c:2847
  #58 0x013849d9 in Ffuncall (nargs=3, args=0x829048) at eval.c:2794
  #59 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8abf0),
      vector=XIL(0xa000000005e8aac8), maxdepth=make_fixnum(19),
      args_template=make_fixnum(514), nargs=2, args=0x829670) at bytecode.c:633
  #60 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8aa98), nargs=2,
      arg_vector=0x829660) at eval.c:2989
  #61 0x01384a33 in Ffuncall (nargs=3, args=0x829658) at eval.c:2796
  #62 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8b048),
      vector=XIL(0xa000000005e8a818), maxdepth=make_fixnum(27),
      args_template=make_fixnum(512), nargs=2, args=0x829d58) at bytecode.c:633
  #63 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8a7e8), nargs=2,
      arg_vector=0x829d48) at eval.c:2989
  #64 0x01384a33 in Ffuncall (nargs=3, args=0x829d40) at eval.c:2796
  #65 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8d5d0),
      vector=XIL(0xa000000005e8d040), maxdepth=make_fixnum(12),
      args_template=make_fixnum(257), nargs=1, args=0x82a380) at bytecode.c:633
  #66 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8d010), nargs=1,
      arg_vector=0x82a378) at eval.c:2989
  #67 0x01384a33 in Ffuncall (nargs=2, args=0x82a370) at eval.c:2796
  #68 0x0137f3e7 in internal_condition_case_n (bfun=0x13847ec <Ffuncall>,
      nargs=2, args=0x82a370, handlers=XIL(0x30),
      hfun=0x105a91d <safe_eval_handler>) at eval.c:1435
  #69 0x0105ab54 in safe__call (inhibit_quit=false, nargs=2,
      func=XIL(0x42cb120), ap=0x82a434 "") at xdisp.c:2807
  #70 0x0105abcd in safe_call (nargs=2, func=XIL(0x42cb120)) at xdisp.c:2822
  #71 0x0105ac21 in safe_call1 (fn=XIL(0x42cb120), arg=make_fixnum(1))
      at xdisp.c:2833
  #72 0x0105e9c7 in handle_fontified_prop (it=0x82a6c0) at xdisp.c:4136
  #73 0x0105d271 in handle_stop (it=0x82a6c0) at xdisp.c:3664
  #74 0x01069335 in reseat (it=0x82a6c0, pos=..., force_p=true) at xdisp.c:6900
  #75 0x0105c4c3 in init_iterator (it=0x82a6c0, w=0x70c0b88, charpos=1,
      bytepos=1, row=0x68432b0, base_face_id=DEFAULT_FACE_ID) at xdisp.c:3265
  #76 0x0105c55b in start_display (it=0x82a6c0, w=0x70c0b88, pos=...)
      at xdisp.c:3281
  #77 0x01091a71 in try_window (window=XIL(0xa0000000070c0b88), pos=...,
      flags=1) at xdisp.c:19056
  #78 0x0108e687 in redisplay_window (window=XIL(0xa0000000070c0b88),
      just_this_one_p=false) at xdisp.c:18480
  #79 0x01086178 in redisplay_window_0 (window=XIL(0xa0000000070c0b88))
      at xdisp.c:16194
  #80 0x0137f1f7 in internal_condition_case_1 (
      bfun=0x108611d <redisplay_window_0>, arg=XIL(0xa0000000070c0b88),
      handlers=XIL(0xc000000005fd6d40), hfun=0x10860da <redisplay_window_error>)
      at eval.c:1379
  #81 0x0108609d in redisplay_windows (window=XIL(0xa0000000070c0b88))
      at xdisp.c:16174
  #82 0x010848b0 in redisplay_internal () at xdisp.c:15642
  #83 0x0108577c in redisplay_preserve_echo_area (from_where=2) at xdisp.c:15995
  #84 0x01019565 in Fredisplay (force=XIL(0)) at dispnew.c:6085
  #85 0x01384f7d in funcall_subr (subr=0x195a0e0 <Sredisplay>, numargs=0,
      args=0x82e128) at eval.c:2867
  #86 0x013849d9 in Ffuncall (nargs=1, args=0x82e120) at eval.c:2794
  #87 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005d746e0),
      vector=XIL(0xa000000005d744f0), maxdepth=make_fixnum(7),
      args_template=make_fixnum(769), nargs=1, args=0x82e638) at bytecode.c:633
  #88 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005d744c0), nargs=1,
      arg_vector=0x82e630) at eval.c:2989
  #89 0x013855f9 in apply_lambda (fun=XIL(0xa000000005d744c0),
      args=XIL(0xc000000000fc65e0), count=10) at eval.c:2926
  #90 0x01382c31 in eval_sub (form=XIL(0xc000000000fc65f0)) at eval.c:2318
  #91 0x0137afa2 in Fprogn (body=XIL(0xc000000000fc6580)) at eval.c:462
  #92 0x013860de in funcall_lambda (fun=XIL(0xc000000000fc55b0), nargs=0,
      arg_vector=0x82ecd8) at eval.c:3060
  #93 0x01384bef in Ffuncall (nargs=1, args=0x82ecd0) at eval.c:2808
  #94 0x0136e970 in Ffuncall_interactively (nargs=1, args=0x82ecd0)
      at callint.c:254
  #95 0x01384e99 in funcall_subr (subr=0x1960720 <Sfuncall_interactively>,
      numargs=1, args=0x82ecd0) at eval.c:2847
  #96 0x013849d9 in Ffuncall (nargs=2, args=0x82ecc8) at eval.c:2794
  #97 0x013830ff in Fapply (nargs=3, args=0x82ecc8) at eval.c:2377
  #98 0x0136f077 in Fcall_interactively (function=XIL(0x4cc4f70),
      record_flag=XIL(0), keys=XIL(0xa000000006803768)) at callint.c:342
  #99 0x01384ff4 in funcall_subr (subr=0x1960740 <Scall_interactively>,
      numargs=3, args=0x82f010) at eval.c:2872
  #100 0x013849d9 in Ffuncall (nargs=4, args=0x82f008) at eval.c:2794
  #101 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005ecd2a8),
      vector=XIL(0xa000000005ecd050), maxdepth=make_fixnum(13),
      args_template=make_fixnum(1025), nargs=1, args=0x82f610) at bytecode.c:633
  #102 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005ecd020), nargs=1,
      arg_vector=0x82f608) at eval.c:2989
  #103 0x01384a33 in Ffuncall (nargs=2, args=0x82f600) at eval.c:2796
  #104 0x01383eec in call1 (fn=XIL(0x3f30), arg1=XIL(0x4cc4f70)) at eval.c:2654
  #105 0x011e09da in command_loop_1 () at keyboard.c:1463
  #106 0x0137f10d in internal_condition_case (bfun=0x11dfd8f <command_loop_1>,
      handlers=XIL(0x90), hfun=0x11df003 <cmd_error>) at eval.c:1355
  #107 0x011df806 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1091
  #108 0x0137e2d8 in internal_catch (tag=XIL(0xdfb0),
      func=0x11df7ca <command_loop_2>, arg=XIL(0)) at eval.c:1116
  #109 0x011df785 in command_loop () at keyboard.c:1070
  #110 0x011dea8b in recursive_edit_1 () at keyboard.c:714
  #111 0x011ded01 in Frecursive_edit () at keyboard.c:786
  #112 0x011d361b in main (argc=2, argv=0xa42848) at emacs.c:2054

  Lisp Backtrace:
  "Automatic GC" (0x0)
  "modify-syntax-entry" (0x824368)
  "perl-quote-syntax-table" (0x824910)
  "perl-syntax-propertize-special-constructs" (0x825050)
  "perl-syntax-propertize-function" (0x825858)
  "syntax-propertize" (0x825e90)
  "internal--syntax-propertize" (0x826458)
  "re-search-forward" (0x8274d8)
  "font-lock-fontify-keywords-region" (0x827c20)
  "font-lock-default-fontify-region" (0x828268)
  "font-lock-fontify-region" (0x8287d8)
  0x7255bc0 PVEC_COMPILED
  "run-hook-wrapped" (0x829050)
  "jit-lock--run-functions" (0x829660)
  "jit-lock-fontify-now" (0x829d48)
  "jit-lock-function" (0x82a378)
  "redisplay_internal (C function)" (0x0)
  "redisplay" (0x82e128)
  "sit-for" (0x82e630)
  "foo" (0x82ecd8)
  "funcall-interactively" (0x82ecd0)
  "call-interactively" (0x82f010)
  "command-execute" (0x82f608)