Mail indexing

[Tim X]

Some advice requested.......

I've been a long time user of Emacs VM for reading my mail and am very
happy with it. However, I find the amount of mail I recieve,
especially at work, is beginning to become a burden when I need to
retrieve an old message or some information, particularly when i can't
remember exactly who the message was from or when it was sent. I have
been using grepmail, but the amount of mail archives I have is
beginning to cause problems with grepmail - particularly with respect
to search time - I also have some other users after this functionality
and grepmail is a bit cumbersome for some of them.

So, my immediate thought was that it was time to look at some of the
data indexing and searching tools out there. 

I started with beagle as I'd heard some good things about it. I've run
it for a week or so and have to admit it isn't half bad. However, it
seems that it really only supports indexing of evolution style mail
files and control files. It won't index VM mail folders. This is a bit
frustrating as beagle seems great with other file types. 

I then thought, OK, I'll find a mail indexing and searching app just
for my VM folders and keep beagle for everything else. Not the
solution I was hoping for, but workable none the less. 

This is where I'm running into difficulties. It seems there are quite
a few options and I'm hoping to get some guideance from the group -
particularly anyone who has gone through this process already. 

My requirements -

1. Just indexing and searching of VM mail folders. 
2. Ideally, the indexing would be automatically updated, similar to
beagle. I currently have a bit over 2Gb of mail archives going back
about 5 years. Having to re-index every night to update for a few new
messages would be annoying and a waste of resources.
3. Searching should support "and", "or" and "not" type modifers.
Phrase based search would be nice, but not critical. 
4. I would like to be able to exclude files easily (such as vm *.spool
files, my procmail log file and backup file etc. 
5. Existing integration with emacs which would make jumping to the
relevant folder and having it opened by VM would be a time saver
6. I don't want to go down the Java route if it can be avoided. I
currently don't run anything else depending on Java and having to
maintain a Java environment just to index my mail archives would be
irritating (it took me 5 years to get away from Java as a developer
and I'm in no hurry to go back!). 

Currently, I'm wondering about two possibilities 

Namazu
Mairix 

or possibly a solution to get beagle to work with VM mail folders (or
mbox folders genreeally). So far, I've only found nasty hacks which
require you to run evolution and click on the new mail folders to have
them indexed etc. 

Any suggestions appreciated.

regards,

Tim

-- 
tcross (at) rapttech dot com dot au