bug#14281: 24.3; replace-match leaves point at wrong place
From:
Barry OReilly
Subject:
bug#14281: 24.3; replace-match leaves point at wrong place
Date:
Fri, 26 Apr 2013 21:19:47 -0400
I've been experiencing an intermittent and difficult to reproduce bug where I use Evil (rev 42737279f1b75ec3731c3f3b0f2c465862159b40) to search and replace in a region and when the bug occurs, replacements are made throughout the buffer and point is left in the wrong place. Reproduction comes in streaks: when I witness it once, I can do it again and again for a while, then it stops reproducing at all. I made progress tracking the cause down such that I can report some findings here. I didn't finish narrowing down the problem because it stopped reproducing.
Description of events at the Lisp level: • Let my-old-string be a string which I've redacted for privacy. It is 14 non regex chars. • Let my-new-string be a redacted string. It is 16 chars, no backslashes. • Evil calls: (perform-replace my-old-string (replace-eval-replacement . my-new-string) nil t nil nil nil 6605 6749) • perform-replace calls: (replace-match my-new-string t nil) match-data="" 6700 [redacted filename]) • Before the call to replace-match, point is 6700 • After the call to replace-match, point is inexplicably 25
Since replace-match is implemented in C, I attached gdb to the running Emacs. I find that when the variable newpoint is assigned by: newpoint = search_regs.start[sub] + SCHARS (newtext); it is 9+16. I can see where 16 comes from, but not 9. sub is 0; I found search_regs.start[0] has the correct value of 6686 for a time prior. I continued to narrow down when search_regs.start[0] changes until the problem stopped reproducing. Here is the stack trace where it goes wrong:
#6 0x00000000004fb13f in signal_after_change (charpos=6686, lendel=14, lenins=16) at insdel.c:2058 #7 0x00000000004fcd87 in replace_range (from=6686, to=<value optimized out>, new=<value optimized out>, prepare=<value optimized out>, inherit=false, markers=true) at insdel.c:1427 #8 0x0000000000516cd9 in Freplace_match (newtext=158062897, fixedcase=<value optimized out>, literal=<value optimized out>, string=1, subexp=<value optimized out>) at search.c:2644 #9 0x0000000000544d8f in eval_sub (form=<value optimized out>) at eval.c:2157 #10 0x000000000054511f in Fprogn (args=<value optimized out>) at eval.c:360 #11 0x0000000000545454 in funcall_lambda (fun=167311462, nargs=5, arg_vector=0x7fff400cb870) at eval.c:3003 #12 0x000000000054686c in apply_lambda (fun=167311446, args=19819474) at eval.c:2887 #13 0x0000000000544a84 in eval_sub (form=<value optimized out>) at eval.c:2218 #14 0x000000000054504f in Fsetq (args=<value optimized out>) at eval.c:429 #15 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #16 0x000000000054511f in Fprogn (args=<value optimized out>) at eval.c:360 #17 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #18 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #19 0x000000000054511f in Fprogn (args=<value optimized out>) at eval.c:360 #20 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #21 0x0000000000544c63 in eval_sub (form=<value optimized out>) at eval.c:2214 #22 0x000000000054511f in Fprogn (args=<value optimized out>) at eval.c:360 #23 0x0000000000547a08 in Fwhile (args=<value optimized out>) at eval.c:936 #24 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #25 0x0000000000546975 in Funwind_protect (args=145892294) at eval.c:1148 #26 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #27 0x000000000054511f in Fprogn (args=<value optimized out>) at eval.c:360 #28 0x0000000000547e97 in FletX (args=131856198) at eval.c:844 #29 0x0000000000544f55 in eval_sub (form=<value optimized out>) at eval.c:2091 #30 0x000000000054511f in Fprogn (args=<value optimized out>) at eval.c:360 #31 0x0000000000545454 in funcall_lambda (fun=137050230, nargs=9, arg_vector=0x7fff400cc2f0) at eval.c:3003 #32 0x0000000000545692 in Ffuncall (nargs=10, args=<value optimized out>) at eval.c:2839 #33 0x0000000000577b1e in exec_byte_code (bytestr=6605, vector=38, maxdepth=11914850, args_template=11914850, nargs=0, args=0x0) at bytecode.c:900 #34 0x00000000005453d7 in funcall_lambda (fun=14279397, nargs=5, arg_vector=0x7fff400cc4e8) at eval.c:3010 #35 0x0000000000545692 in Ffuncall (nargs=6, args=<value optimized out>) at eval.c:2839 #36 0x0000000000545cc4 in Fapply (nargs=2, args=0x7fff400cc590) at eval.c:2312 #37 0x0000000000545ee0 in apply1 (fn=97848594, arg=<value optimized out>) at eval.c:2546 #38 0x0000000000541564 in Fcall_interactively (function=97848594, record_flag=11914850, keys=11950037) at callint.c:377 #39 0x00000000005457f7 in Ffuncall (nargs=2, args=<value optimized out>) at eval.c:2785 #40 0x0000000000577b1e in exec_byte_code (bytestr=6605, vector=33, maxdepth=11914850, args_template=11914850, nargs=0, args=0x0) at bytecode.c:900 #41 0x00000000005453d7 in funcall_lambda (fun=19430901, nargs=3, arg_vector=0x7fff400cc8b0) at eval.c:3010 #42 0x000000000054686c in apply_lambda (fun=19430901, args=158068849) at eval.c:2887 #43 0x0000000000544a84 in eval_sub (form=<value optimized out>) at eval.c:2218 #44 0x0000000000546edd in Feval (form=169173046, lexical=<value optimized out>) at eval.c:2005 #45 0x00000000005457e0 in Ffuncall (nargs=2, args=<value optimized out>) at eval.c:2781 #46 0x0000000000577b1e in exec_byte_code (bytestr=6605, vector=33, maxdepth=11914850, args_template=11914850, nargs=0, args=0x0) at bytecode.c:900 #47 0x00000000005453d7 in funcall_lambda (fun=19550949, nargs=1, arg_vector=0x7fff400ccd48) at eval.c:3010 #48 0x0000000000545692 in Ffuncall (nargs=2, args=<value optimized out>) at eval.c:2839 #49 0x0000000000545e47 in Fapply (nargs=2, args=0x7fff400ccd40) at eval.c:2259 #50 0x0000000000545ee0 in apply1 (fn=13903650, arg=<value optimized out>) at eval.c:2546 #51 0x0000000000541564 in Fcall_interactively (function=13903650, record_flag=11914850, keys=11950037) at callint.c:377 #52 0x00000000005457f7 in Ffuncall (nargs=4, args=<value optimized out>) at eval.c:2785 #53 0x0000000000545a74 in call3 (fn=<value optimized out>, arg1=<value optimized out>, arg2=11914850, arg3=11914898) at eval.c:2603 #54 0x00000000004e5402 in command_loop_1 () at keyboard.c:1587 #55 0x00000000005441a6 in internal_condition_case (bfun=0x4e5040 <command_loop_1>, handlers=11966498, hfun=0x4dc150 <cmd_error>) at eval.c:1289 #56 0x00000000004dbc9a in command_loop_2 (ignore=<value optimized out>) at keyboard.c:1168 #57 0x000000000054429a in internal_catch (tag=<value optimized out>, func=0x4dbc80 <command_loop_2>, arg=11914850) at eval.c:1060 #58 0x00000000004dc3f0 in command_loop () at keyboard.c:1147 #59 recursive_edit_1 () at keyboard.c:779 #60 0x00000000004dc53a in Frecursive_edit () at keyboard.c:843 #61 0x00000000004d4146 in main (argc=<value optimized out>, argv=0x7fff400cd538) at emacs.c:1528
Before replace_match's call to signal_after_change, search_regs.start[0] is correct, after signal_after_change returns, it is incorrect.
Looking at the code, I see what I think are callbacks to Lisp related to overlays. It's notable that Evil uses overlays to display the search and replace as the user types the command in, showing the old string striked out the old with the new string alongside. Is it possible the invoked hooks could replace the match data and thus cause search_regs to have wrong values upon return? Are there other ideas about what's going on? The next time I witness the bug, I'll narrow down where in signal_after_change the value becomes wrong.
In GNU Emacs 24.3.2 (x86_64-unknown-linux-gnu, GTK+ Version 2.10.4) of 2013-03-11 on psd15 Windowing system distributor `The X.Org Foundation', version 11.0.70101000 System Description: Red Hat Enterprise Linux Client release 5.4 (Tikanga)
Important settings: value of $LANG: en_US.UTF-8 value of $XMODIFIERS: @im=none locale-coding-system: utf-8-unix default enable-multibyte-characters: t
Major mode: Fundamental
Minor modes in effect: diff-auto-refine-mode: t shell-dirtrack-mode: t global-whitespace-mode: t global-cedet-m3-minor-mode: t global-semantic-mru-bookmark-mode: t global-semanticdb-minor-mode: t global-semantic-idle-scheduler-mode: t semantic-mode: t global-ede-mode: t evil-mode: t evil-local-mode: t global-undo-tree-mode: t undo-tree-mode: t show-paren-mode: t delete-selection-mode: t global-auto-revert-mode: t tooltip-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t column-number-mode: t line-number-mode: t transient-mark-mode: t
Recent input: ' ' ' ' ' ' ' / h e r e SPC 1 3 <return> N N n ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ; j k k k k k k k k k k k k k k k k k k k k k k k k k k k k V y f j j j j j j <return> / n e w t e x t <return> f j <return> k k V y f j <return> V y f j <return> j j j k f j j j <return> f <return> ; ' ' ' j V y C-c V y C-c V y / s u b <return> ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' k k k k k k k k k k k k n 0 f <return> j k f j j j j j j j H <return> f <return> f j <return> j k W W W W h h h h h h h h h h h h h h h h h v l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l y M-x r e p o r t - <tab> <return>
Recent messages: Undo branch point! Commands: d, s, x, u; f, o, 1, 2, m, v; ~, %; q to quit; ? for help. [5 times] Mark set [7 times] Commands: d, s, x, u; f, o, 1, 2, m, v; ~, %; q to quit; ? for help. Mark set Commands: d, s, x, u; f, o, 1, 2, m, v; ~, %; q to quit; ? for help. [5 times] Mark set [2 times] Commands: d, s, x, u; f, o, 1, 2, m, v; ~, %; q to quit; ? for help. [2 times] Mark set Commands: d, s, x, u; f, o, 1, 2, m, v; ~, %; q to quit; ? for help. [2 times]