Re: [Ifile-discuss] Re: Maildir format

[Jack Bertram]

* Brett Nemeroff <address@hidden> [030514 15:43]:
> So I'm looking at Jack's original refile.learn script on my server. I'm
> going to try to comment it so I can understand what's going on. If I'm
> wrong, can you please correct me? I'd rather understand what's going on
> than just simply ask you guys for the new scripts....

Oh dear - my quick hacks exposed and dissected...

> ------------ refile.learn --------------------------
> #!/bin/bash
> source /etc/profile
> source $HOME/.bashrc
> 
> MAILDIR=$HOME/Maildir/
> BACKUP=$HOME/.backupmail
> 
> #If $BACKUP dir dosen't exist, make it
> [ ! -d $BACKUP ] && mkdir $BACKUP
> 
> #Step through the $MAILDIR file by file (
> #QUESTION: Will this work for MAILDIR since it uses [dot]MailboxName format?

No

> # Perhaps it should be:
> # for mbox in $(ls -a $MAILDIR/)

Yes

> # However that has extra stuff as well in it (. and ..)

You need to modify the if statement...

> for mbox in $(ls $MAILDIR/)
> do
> # If the current Mailbox exists then...

This currently tests if the directory entry is a _file_ and exists (to
ensure it's a mbox, not a directory).  You need to modify it to look for
your maildir names.

Something like (untested):

>         if [ -f $MAILDIR/$mbox ]

if [ -d $MAILDIR/$mbox && $mbox -ne '.' && $mbox -ne '..' ] 

>         then
> # Create a Backup
> # This won't work in Maildir since Maildir uses
> # "cur" "new" and "tmp" folders. A recursive copy should fix this? #
> # Perhaps: cp -aR $MAILDIR/$mbox $BACKUP/$mbox || exit

Yes, something like this would work.  However you would not be replacing
the existing contents of the backup, which you might want to do.
(Otherwise you won't ever remove deleted messages from your backups)

You could do

  rm -rf $BACKUP/$mbox && cp -aR $MAILDIR/$mbox $BACKUP/$mbox || exit

>                 cp $MAILDIR/$mbox $BACKUP/$mbox || exit
> # Reprocess mailboxes, creating newly processed mboxes with a .new extension
> # This entire structure seems to need to be replaced. During ther
> mid-stage # of the reclassification, it should write a <message>.new
> instead of a
> # <folder>.new, right? We'll have to take a look at the procmail file
> # and see what's going on there as well...
>                 formail -s procmail -m FOLDER=$mbox

This is where you need to do something like

  for msg in $mbox/cur/*; do cat $msg | procmail -m MSG=$msg FOLDER=$mbox 
$HOME/bin/refile.learn.rc; done

Note that this may not work for very large Maildirs (thousands of
messages) as the command line will be too long.  In this case you would
have to do something more like (untested)

Actually, you also need to pass the name of the message file to procmail in
the environment, as you won't be using procmail delivery. Done above.

ls $mbox/cur/* | xargs cat | procmail ...

> $HOME/bin/refile.learn.rc <
> $MAILDIR/$mbox
> # If there is a new MBOX and if it's any different, then overwrite the
> # old MBOX

Here you'd do something like

# Find changed messages
 
  for newmsg in $MAILDIR/cur/*.new;
  do

# Find the name of the non-changed message
 
    msg = `echo $newmsg | sed -e 's/\.new//'` 

# If it's different, replace it with the changed one

    diff $newmsg $msg > /dev/null 2>/dev/null || mv $newmsg $msg
    [ -f $newmsg ] && rm -r $newmsg
  done

>                 if [ -f $MAILDIR/$mbox.new ]
>                 then
>                         diff $MAILDIR/$mbox.new $MAILDIR/$mbox >/dev/null
> 2>/dev
> /null || mv $MAILDIR/$mbox.new $MAILDIR/$mbox
> # If there was a .new MBOX, get rid of it, we're done with it.

Well, it's more like "If there's still a new mailbox for some reason,
then let's delete it".  It should have gone in the last step.

> 
>                         [ -f $MAILDIR/$mbox.new ] && rm -f $MAILDIR/$mbox.new
>                 fi
>         fi
> 
> 
> done
> --------------------------------------
> Question, I'm assuming here that the script waits for the execution of the
> procmail command to complete before continuing?? Seems like it would have
> to for this to work...

Yes

> Ok, on the RC script, I started making some Maildir changes...
> --------------refile.learn.rc---------------
> SHELL = /bin/sh
> 
> MAILDIR=$HOME/Maildir/
> 
> #Put stuff already filed by ifile into the "new" (temp) folder

# Put stuff already _correctly_ filed by ifile into the new mbox.  

You don't need this for Maildir, as you don't need to change correct
messages.  Instead

:0:
* $ ^X-Ifile-Hint: $FOLDER
>/dev/null

> :0:
> * $ ^X-Ifile-Hint: $FOLDER
> .$FOLDER.new/
> 
> # Put stuff refiled by ifile into the temp folder
> # Question, why would this rule catch anything? Shouldn't the previous rule
> # catch everything for here? Or are we just populating $FOLDER?

This rule catches stuff which has already been _relearned_ to the new
mbox.  Again, you don't need it:

:0:
* $ ^X-Ifile-Learned-To: $FOLDER
>/dev/null

> :0:
> * $ ^X-Ifile-Learned-To: $FOLDER
> .$FOLDER.new/


> # relearn message
> :0 wc
> * ^X-Ifile-Hint: \/.+
> | $HOME/bin/ifile.relearn.message $MATCH $FOLDER 2>/dev/null
> 
> # add (change?) heading to reflect that it's been learned
> # I have no idea how this changes for Maildir... Arn't individual mesages
> # passed right here from procmail? why is a formail necessary?

formail is what inserts the new header...

>         :0 Af
>         | formail -I "X-Ifile-Learned-To: $FOLDER"
> 
> # I'm not sure about this guy.. maybe file refiled stuff into the temp folder

...and this puts it into the mbox.  You probably want to "cat" it onto
the new message (so not really using procmail delivery)

>         :0 A:
>         .$FOLDER.new/

so replace with

:0 A:
| cat > .$FOLDER/cur/$MSG.new

> 
> # Catch unfiled emails (not touched by ifile), or just "other" emails..

Not necessary for Maildir - remove

> :0:
> .$FOLDER.new/
> --------------------------------------------
> 
> 
> So can you guys give me a little direction based on my comments. I'm going
> to spend a little time looking over some procmail docs to see what I can
> figure out.. but mainly I have questions about how to process the
> individual files and to recurse (not sure if recursing is entirely
> necessary, or just going into those "new" "cur" and "tmp" dirs..) into the
> subdirs in the Maildir format.. So mainily what to do with the formail
> statements.. You gave me one of the formail replacements, what do I need
> to do with this other one??

Hope this helps.  If you want to try any of this, make sure you do it
with a backup (small, for testing) copy of your mail spool and a backup
copy of your ifile database, and check very carefully what it's doing at
each step.

> Thanks!!
> Brett

Jack