Re: [VM] searching in mime encoded email

[John Hein]

Uday Reddy wrote at 15:51 +0000 on Jan 19, 2012:
 > address@hidden writes:
 > > Sorry, my message was missing a critical phrase - I want the relevant
 > > parts of the message decoded before the messages are saved.  Right now I
 > > do this several times a day using a couple of crude keyboard macros, so I
 > > expect I can fully automate this but perhaps this has already been done?
 > 
 > I see a function called vm-save-message-preview in vm-rfaddons.el.
 > Please try it.

That doesn't seem to work for mime parts, but it does save the message
decoded if the entire message was, for instance, marked as encoded
with base64.

It does not save it as a message in a vm folder or even
an mbox file (no 'From ').

If I save attach a utf-8 file (used
http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt-8-demo.txt)
and then use vm-save-message-preview to save it to
a new folder, then manually add a 'From ' line, then view the
folder with vm (in emacs -nw in a utf-8 compatible terminal),
it's quite happy.  Content-* headers are:

Content-Transfer-Encoding: base64    [1]
Content-Type: text/plain

[1] even though it's not base64 anymore after vm-save-message-preview

Viewing that folder in thunderbird (3.6) is not happy [2] unless I
change the Content-Type to:

Content-Type: text/plain; charset="utf-8"

So thunderbird would be okay with it if the message marked the
charset properly.

Both thunderbird and vm can search the individual message and find
utf-8 characters (vm-isearch-presentation in vm) in the raw utf-8
stream (and both can search the individual message as well when it's
base64 encoded).

And forwarding the message with the decoded mime through a local
sendmail (freebsd) was fine.  I can imagine it not working with some
mailers that can't handle 8bit.


 > More generally, I am thinking that there is no reason why we can't
 > have VM folders stored in some other character set, other than
 > US-ASCII, e.g., UTF-8.  Those folders won't be interoperable with
 > other mail clients, but do we care about that really?  That should
 > be a big win for people that need to use international character
 > sets regularly.  MIME-decoded text can then be stored directly into
 > folders and all normal Emacs searching functions will work.
 > 
 > This needs some thought and discussion, and it will need some
 > amount of careful reengineering effort.  The assumption about 7-bit
 > US-ASCII is probably pervasive in a lot of VM code.  So, it will
 > need extensive testing, and I would need people that are willing to
 > participate in that.  There could be possibility of corruption of
 > mail folders.  They would need to back up email carefully.  But, it
 > could be a big win in the long run.

If we do remove all seat belts and allow saving in utf-8 or similar,
we should make sure there's a way to re-encode back to safe encodings
easily.  Maybe unsafe encoding could go to a special folder name??  As
mentioned above, forwarding such messages could be an issue with some
mailers (MTAs).

I tend to think that the tools we use in vm should be able to operate
on these [base64 or uuencode] encoded blocks (e.g., have folder
searches be able to reach into decoded mime blocks) rather than trying
to re-encode them except in carefully controlled situations.  The
latter is useful, but natively grokking base64 seems like a bigger
win.


[2] partial unhappy thunderbird image attached...