Re: RMAIL slows

[Robert J. Chassell]

Stefan Monnier <address@hidden> wrote, 

   ... re_match_2_internal is a function that can easily take an
   insane amount of time ...

   So a good thing to do is to look at the regexp that's being matched
   (typically: look up the backtrace to see the value of the `string' argument
   to search_buffer, then do "print string" and then "xstring" to see the
   actual string).  

Thank you.  This and Kim's message are what I need.  I cannot figure
out why things go wrong.  They look OK to me.

    ... let the code run until the end of re_match_2_internal.  If
    it's immediate it means the problem is maybe not in the time taken
    in each call to re_match_2_internal, but rather in the number of
    calls.

When I evoke `finish' when the program has stopped at
re_match_2_internal, it runs fairly quickly to then stop in
re_search_2 in regex.c, line 4338, at the second line of 

      if (val >= 0)
        return startpos;

even though I do not have a break point there.  (The only breakpoint
for this run is in re_match_2_internal at regex.c:4828)

I am returning fairly quickly from `finish' so the problem may be in
the number of calls, not in re_match_2_internal.  I am deleting 100
messages per batch.  The point is, such a deletion is quick for a
fresh instance of Emacs but slow after that instance of Emacs has been
running for several days.


In any event, here first is     search_buffer  then  re_match_2_internal 


1. Here is the backtrace from a break for     search_buffer
   ( break search.c:1051 ), followed by
        (gdb) c
        (gdb) print string
        (gdb) xstring
        (gdb) finish

#0  search_buffer (string=189937, pos=137573113, pos_byte=137573161, 
lim=137573113, lim_byte=1, n=1, RE=0, trt=135730036, inverse_trt=-1073747680, 
posix=188026) at search.c:1051
#1  0x08159be9 in search_command (string=144556139, bound=137573113, 
noerror=137573161, count=188026, direction=1, RE=1, posix=0) at search.c:939
#2  0x0815c1fe in Fre_search_forward (regexp=188026, bound=188026, 
noerror=188026, count=188026) at search.c:2160

(gdb) c
Continuing.

Breakpoint 3, search_buffer (string=189937, pos=137573113, pos_byte=137573161, 
lim=137573113, lim_byte=1, n=1, RE=0, trt=135730036, inverse_trt=-1073747680, 
posix=188026) at search.c:1051

(gdb) print string
$1 = 189937

(gdb) xstring
$2 = (struct Lisp_String *) 0x2e5f0
Cannot access memory at address 0x2e5fc

(gdb) finish
Run till exit from #0  search_buffer (string=189937, pos=137573113, 
pos_byte=137573161, lim=137573113, lim_byte=1, n=1, RE=0, trt=135730036, 
inverse_trt=-1073747680, posix=188026) at search.c:1051
search_command (string=144556139, bound=137573113, noerror=137573161, 
count=188893, direction=1, RE=1, posix=0) at search.c:947
Value returned is $3 = 188893

(gdb) pp string
"\\<\\(ftp://\\|http://\\|gopher://\\|news:\\|nntp:\\|telnet://\\|wais://\\|file:/\\|prospero:\\|z39.50s:\\|z39.50r:\\|cid:\\|mid:\\|vemmi:\\|service:\\|imap:\\|nfs:\\|acap:\\|rtsp:\\|tip:\\|pop:\\|data:\\|dav:\\|opaquelocktoken:\\|sip:\\|tel:\\|fax:\\|modem:\\|ldap:\\|https://\\|soap.beep:\\|soap.beeps:\\|urn:\\|go:\\|afs:\\|tn3270:\\|mailserver:\\|snews:\\)[^]
    
 \"'()<>[^`{}]*[^]      
 \"'()<>[^`{}.,;]+"

(gdb) xbacktrace
"re-search-forward"
"goto-address-fontify"
"goto-address"
"run-hooks"
"rmail-show-message"
"rmail-summary-goto-msg"
"rmail-summary-delete-forward"
"call-interactively"





2. Then I put in a break for     re_match_2_internal 

    (gdb) break regex.c:4828

    and continued.  Here is the backtrace and the results of 
        (gdb) print string1
        (gdb) xstring
        (gdb) finish

    The match is to a spam mail that is in the test mbox I am using.

#0  re_match_2_internal (bufp=0x83298cc, string1=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size1=0, string2=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size2=1911, pos=867, regs=0x414, stop=0) at regex.c:4835
#1  0x08164102 in re_search_2 (bufp=0x83298cc, str1=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size1=0, str2=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size2=1911, startpos=872, range=1039, regs=0x8320268, 
stop=1911) at regex.c:4328
#2  0x0815a3e5 in search_buffer (string=140965112, pos=188893, pos_byte=867, 
lim=189937, lim_byte=189937, n=1, RE=1, trt=142892020, inverse_trt=142937924, 
posix=1084985957) at search.c:1138

(gdb) print string1
$4 = (const unsigned char *) 0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"...

(gdb) xstring
Argument to arithmetic operation not a number or boolean.

(gdb) finish
Run till exit from #0  re_match_2_internal (bufp=0x83298cc, string1=0x40ab9265 
"From: \"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size1=0, string2=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size2=1911, pos=867, regs=0x414, stop=0) at regex.c:4835
re_search_2 (bufp=0x83298cc, str1=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size1=0, str2=0x40ab9265 "From: 
\"\201\274\201\256\201\301\201\370\201\277\201\3549\" 
<address@hidden>\nReply-To: 
\"\201\261\201\350\201\271\201\316\201\305\201\3029\" <address@hidden>\nTo: 
address@hidden: 
\201\301\201\367\201\300\201\345\201\300\201\316,\201\260\201\370\201\271\201\253\201\277\201\370\201\300\201\273\201\300\201\247\201\307\201\321\201\267\201\320\201\300\201\316\201\306\201\367,\201\303\201\326\201\260\201\355\201\300\201\307\201\307\201\375\201\305\201\303
 \201\277\201\3710"..., size2=1911, startpos=872, range=1039, regs=0x8320268, 
stop=1911) at regex.c:4337
Value returned is $5 = -1


What should I do next?

-- 
    Robert J. Chassell                         
    address@hidden                         GnuPG Key ID: 004B4AC8
    http://www.rattlesnake.com                  http://www.teak.cc