Re: New question about interaction between spamass-milter and clamav.

[Steven W. Orr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/07/09 13:21, quoth EASY address@hidden:
> On Tue, 2009-07-07 at 13:06 -0400, Steven W. Orr wrote: On 07/07/09 13:00, 
> quoth EASY address@hidden:
>>>> On Tue, 2009-07-07 at 12:42 -0400, Steven W. Orr wrote: I need some 
>>>> help with whether I'm doing something wrong.
>>>> 
>>>> Here's my setup:
>>>> 
>>>> I have sendmail-8.14.3 spamass-milter-0.3.1-13 (set to reject) 
>>>> spamassassin-3.2.5
>>>> 
>>>> All was good in the world, but slowly the spam that got through was 
>>>> creeping up.
>>>> 
>>>> I found that if I added clamav-milter that a lot of the stuff would 
>>>> get caught. I came to this list a while back to ask about the pros 
>>>> and cons of whether the clamav-milter should go before or after 
>>>> spamass-milter. I decided on doing clamav-milter first. Both milters 
>>>> were set to reject.
>>>> 
>>>> Then the false negs started climbing again and I heard about all the 
>>>> clamav addons that were available via scamp from
>>>> 
>>>> https://sourceforge.net/projects/scamp/
>>>> 
>>>> That made a huge difference and life was good again.
>>>> 
>>>> Again, things were escalating and I decided that the real problem is 
>>>> that every time clamav rejects something, spamassassin doesn't learn 
>>>> from it. The SA AWL and the bayes tables never get informed except 
>>>> for the false negs that get through that get fed to sa-learn --spam. 
>>>> I thought that it would make more sense for SA to run the clam test 
>>>> itself. I looked around and sure enough I found clamav.pm  which is a
>>>>  plugin for SA.
>>>> 
>>>> So where I am now is that I have eliminated the clamav-milter, and 
>>>> I'm running a clamd so that clamscam works, and I installed the 
>>>> clamav plugin to SA. The sendmail incantation I'm using is this 
>>>> standard one:
>>>> 
>>>> INPUT_MAIL_FILTER(`spamassassin', 
>>>> `S=local:/var/run/spamass-milter/spamass-milter.sock, F=, 
>>>> T=C:15m;S:4m;R:4m;E:10m')dnl
>>>> 
>>>> and the params for running spamass-milter are
>>>> 
>>>> -m -u steveo -r 5 -d misc -i 192.168.0.1/24 -i 127.0.0.1/24
>>>> 
>>>> I can't believe you've read this far. But if you got here then I can 
>>>> now ask my question:
>>>> 
>>>> Why is it that I am now seeing  my server asking for retries? I am 
>>>> getting rejects like I expect to get (and like I got before). For 
>>>> example,
>>>> 
>>>> Jul  5 05:17:17 saturn spamass-milter[3478]: queueid=n659HG7c007407 
>>>> Jul  5 05:17:18 saturn sendmail[7407]: n659HG7c007407: 
>>>> from=<address@hidden>, size=2174, class=0, nrcpts=1, 
>>>> msgid=<address@hidden>, proto=ESMTP, 
>>>> daemon=MTA, relay=sombody_good [good.guy.we.like] Jul  5 05:17:18 
>>>> saturn sendmail[7407]: n659HG7c007407: Milter add: header: 
>>>> X-Virus-Scanned: ClamAV 0.94.2/9538/Fri Jul  3 10:27:11 2009 on 
>>>> myserver.syslang.net Jul  5 05:17:18 saturn sendmail[7407]: 
>>>> n659HG7c007407: Milter add: header: X-Virus-Status: Infected with 
>>>> Sanesecurity.Spam.10285.UNOFFICIAL Jul  5 05:17:19 saturn 
>>>> sendmail[7407]: n659HG7c007407: Milter: data, reject=554 5.7.1 virus 
>>>> Sanesecurity.Spam.10285.UNOFFICIAL detected by ClamAV - 
>>>> http://www.clamav.net Jul  5 05:17:19 saturn sendmail[7407]: 
>>>> n659HG7c007407: to=<address@hidden>, delay=00:00:01, 
>>>> pri=32174, stat=virus Sanesecurity.Spam.10285.UNOFFICIAL detected by 
>>>> ClamAV - http://www.clamav.net
>>>> 
>>>> 
>>>> But now I'm also seeing retries when clamav is deciding that the 
>>>> status is unknown. Somehow, spamass-milter is returning a 4.5.1
>>>> 
>>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: 
>>>> from=<address@hidden>, size=3879, class=0, nrcpts=1, 
>>>> msgid=<address@hidden>,
>>>>  proto=ESMTP, daemon=MTA, relay=201-94-178-5.jau.flash.tv.br 
>>>> [201.94.178.5]
>>>> 
>>>> 
>>>> 
>>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: Milter add: 
>>>> header: X-Virus-Scanned: ClamAV 0.94.2/9538/Fri Jul  3 10:27:11 2009 
>>>> on myserver.syslang.net
>>>> 
>>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: Milter add: 
>>>> header: X-Virus-Status: Unknown
>>>> 
>>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: Milter: data,
>>>>  reject=451 4.3.2 Please try again later
>>>> 
>>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: 
>>>> to=<address@hidden>, delay=00:00:01, pri=33879, stat=Please try
>>>>  again later
>>>> 
>>>> I'm not sure that I have a complaint. It's just that I did not have 
>>>> any notion that spamass-milter had the ability to do anything but 
>>>> either accept or reject with a 500 series code and I wanted to 
>>>> understand what was going on. I figured that SA only returns an 
>>>> assessment, but it's spamass-milter that does the actual rejecting.
>>>> 
>>>> Can someone explain this? (And again, thanks for reading.)
>>>> 
>>>> 
>>>>> 
> 
>>>> It makes sense to virus scan first and drop. In an ideal world you 
>>>> should be taking out as much 'obvious' rubbish as early as you can in
>>>>  the SMTP stage. Only give Spamassassin anything that other anti UCE 
>>>> methods cannot latch on to. Spamassassin takes some tweaking. I won't
>>>>  use the Bayes feature at all - but that is a personal thing. You
>>>> have to add rules that meet your requirements and see the kind of
>>>> trends you have. Some examples of the false positives put up at : 
>>>> http://pastebin.com/ may get you some suggestions.
> Ok, but you're actually not on the same subject as what I'm asking: How is 
> it possible for spamass-milter to returns a 400 series retry request? I 
> have it set to either reject if the score is greater than 5 or accept.
> 
> I am NOT asking whether I should do virus checking first. I'm also not 
> asking for how to fine tune SA. I just don't see how these retries are 
> possible given the way it's configured.
> 
> 
> My apologies. I did not spot the 400 question. My inclination is this will 
> be specific to the MTA. It is the MTA that does the 4xx (defer) or 5xx 
> (reject). The milter simple says 'yes' or 'no'. The only reason I could see
>  a 451 and that would be if the milter was unavailable (misconfigured or 
> down). I've seen this with Postfix when the milter is unreachable. (Had 
> some issues with permissions on the unix socket). My guess is sendmail has 
> a similar mechanism.

> If you are sure the milter is up and running and you can feed a test telnet
>  message all the way through, then somewhere your MTA is set to give a 
> temporary fail rather than permanent.

Excellent! We're back on track. The milter is up and running. The spamd is
running and clamd is running. What I see in the log files are 100% consistent:

If SA calls the CLAMAV plugin and it's clean then no score is added because of
 the plugin. But then I'm left with two possibilities. The first is that the
milter rejects because the clamav plugin said that it is a virus. The second
is like this:

Jul  5 09:37:03 saturn sendmail[17349]: n65Db1c4017349: Milter add: header:
X-Virus-Status: Unknown
Jul  5 09:37:03 saturn sendmail[17349]: n65Db1c4017349: Milter: data,
reject=451 4.3.2 Please try again later

So someone is saying unknown. (That's the plugin) That unknown is then being
reported back by spamass-milter with a 451 and then from there it's somehow
turned into a "4.3.2 Please try again later".

It's not making sense to me, if for no other reason that the milter is
supposed to only either accept or reject based on whether the score is greater
than the threshold (or not).

Does this help?

- --
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkpTkNIACgkQRIVy4fC+NyQpMQCfZ2n9QIO19mDofgMN3bjDjBJc
dakAnifGtP5BiKjsy+VHFEOeUH6OE0JZ
=vyEW
-----END PGP SIGNATURE-----