Re: spam handling in No Gnus 0.3

[Ted Zlatanov]

On 19 Mar 2006, alberto.l@nospam.net wrote:

> I have recently upgraded to No Gnus 0.3 and I have some comments about
> spam handling.
>
> I use the nnfolder backend for e-mail, spam-split with bogofilter, and
> gnus-registry.  New messages classified as ham appear in
> mail.incoming.  If I find there spam, then I mark it as spam and, on
> exiting, the message is registered as spam with bogofilter, and moved
> to mail.spam.save.
>
> Later on I double check mail.spam.save (a
> gnus-group-spam-classification-spam group), where I can mark ham
> erroneously classified as spam.  On exiting, spam is deleted, and ham
> is unregistered as spam, and registered as ham with bogofilter.
>
> The first problem I find is that when spam messages are deleted from
> mail.spam.save, they are UN-registered as spam with bogofilter, which
> is obviously undesirable.
>
> I have checked the elisp code in spam.el.  The summary-prepare-exit
> hook routine checks what marks changed in mail.spam.save, and for each
> article whone mark changed from spam to ham (AND viceversa), and was
> registered as spam, it is unregistered as spam.  The articles I marked
> as spam have no mark when the group is entered, then they are
> automatically marked as spam with a summary-prepare hook routine
> (because the group is classified as spam).  When checks are done on
> exiting, all spam articles unchanged by the user are considered
> "changed" because their mark was (automatically) changed.  And, even
> if the mark says "spam", they are unregistered as spam: this appears
> plain wrong to me.
>
> It is quite easy to fix the above, one has to move
> "spam-mark-junk-as-spam-routine" to before (rather than after)
> initializing the "spam-old-articles" in the prepare-summary spam hook:
>

(defun spam-summary-prepare ()
  (spam-mark-junk-as-spam-routine)
  (setq spam-old-articles
  (list (cons 'ham (spam-list-articles gnus-newsgroup-articles 'ham))
        (cons 'spam (spam-list-articles gnus-newsgroup-articles 'spam)))))

> I recommend adopting the above modification.

The whole point of the order is that old articles are from before
unseen articles got marked as spam.  So I don't think this fix will do
it.

Could you use the CVS version of Gnus, and we can start thinking of a
solution against it?

> I use a second group mail.spam.train to train bogofilter both with ham
> and spam messages.  The group is "gnus-group-spam-classification-ham".
> On exiting, articles marked as ham are registered as ham with
> bogofilter.  Articles marked as spam are registered as spam with
> bogofilter and automatically moved to mail.spam.save.
>
> The problem is that ham articles are not moved or deleted, and when I
> re-enter the group in order to expire the article (I have immediate
> expiration) to avoid double registrations, on exiting the ham articles
> are unregistered as ham, which is again quite undesirable.
>
> This happens because, in the summary-prepare-exit spam routine, the
> expired articles have the "expirable" mark, which is not a ham mark,
> thus the code judges that their classification has changed, and since
> gnus-registry remembers they were registered as ham, they are
> unregistered.  Something similar happens if one explicitly expires
> spam in spam groups.  There are two possible solutions:
> - add the expirable mark in the list of both ham and spam marks
> - change the code, preventing expirable articles to be considered as
> articles where the user changed their ham/spam classification.
>
> I think that the second solution is more appropriate: when I expire
> articles I mean to delete them, and not to change their ham/spam
> classification.  In order to do that, spam.el has to be changed as
> follows:
>
***************
*** 1343,1352 ****
        (dolist (backend (spam-backend-list))
          (let (unregister-list)
            (dolist (article changed-articles)
              (let ((id (spam-fetch-field-message-id-fast article)))
                (when (spam-log-unregistration-needed-p
                       id 'process classification backend)
!                 (push article unregister-list))))
            ;; call spam-register-routine with specific articles to unregister,
            ;; when there are articles to unregister and the check is enabled
            (when (and unregister-list (symbol-value backend))
--- 1343,1355 ----
        (dolist (backend (spam-backend-list))
          (let (unregister-list)
            (dolist (article changed-articles)
+             ;; do not unregister articles marked as expirable
+             (unless (eq gnus-expirable-mark
+                         (gnus-summary-article-mark article))
              (let ((id (spam-fetch-field-message-id-fast article)))
                (when (spam-log-unregistration-needed-p
                       id 'process classification backend)
!                 (push article unregister-list)))))
            ;; call spam-register-routine with specific articles to unregister,
            ;; when there are articles to unregister and the check is enabled
            (when (and unregister-list (symbol-value backend))

Expirable articles should probably be ignored, so I think this
modification is OK.  Does anyone else see a problem?

Ted