Re: monit SMTP

[Jan-Henrik Haukeland]

This reminds me about jwz's who says that:

 ``Every program attempts to expand until it can read mail. Those
 programs which cannot so expand are replaced by ones which can.''

Adding our planned mail extensions to monit is our contribution to
this law :-)

But seriously, thank you for a very clear mail with many good
arguments. We _need_ corrections and discussions like this and when
reading your mail I found myself nodding my head more than a few
times. The bottom line is that I agree with you, except on a few
important issues.

First, I'm convinced that we should *not* attempt to implement more
features around the already existing monit SMTP implementation.
Particularly *not* implement mail queue and mail spooling. This will
involve some non-trivial (thread) code and will certainly add more
complexity to the code and add (many) more possibly fail points. I
agree with you that monit should be as simple as possible and as
orthogonal as possible. That is, monit should not include (much) other
functionality than what is needed for it to fullfill it's goal, which
is: monitoring and mending services on a Unix system.

But, I disagree that monit should not implement it's own SMTP
agent. My arguments for this are more or less the same as Christian's
and Martin's but allow me to clarify; If we where about to start
implementing monit now I would agree that using an external program to
handle mail would seem like a good idea, but monit's SMTP agent has
been around from one of the first versions (i.e. it's been part of
monit for many years) and it works! (You know the saying, if it works,
don't change it).

Besides, I cannot see that the very simple operation of opening up a
socket and sending a mail through the SMTP protocoll is more error
prone than calling an external program through execv() and send
mail. The network code in monit is simple, effective and orthogonal
and has not have any problem. (The http bug recently discovered was a
protocol bug not a fault of the network code.)

I do agree with you when you point out that mail is unreliable and
also that it is not very important to ensure that alert mails reach
it's destination:

> > This way we will not loose messages in the case of temporary
> > problem (for example network outage).
>
> But do we need them then?  Isn't logging for that?

In other words the current SMTP agent in monit is good
enough. (Altough I would like to change the SMTP protocol
implementation to use a simple state-machine so it will be more
failsafe, the current implementation is just one glorious write
loop. Even if simplicity is important, there are limits :)

Another argument mentioned before, but one I would like to stress, is
that our design philosophy is that monit should be as autonomous as
possible. The rationale for this is (as mentioned many times before)
that monit should be easy to setup and run. Users should not need to
download and configure many external components. By having a very low
complexity on setting up and run monit means that configure errors are
much less likely to occur.

Finally, an interesting point is that m/monit is sailing up to become
a good alternative to take over many of the more enterprise issues,
such as reporting and statistics and so on. It's not critical that
m/monit runs all the time but it is very critical that monit
does. This means that we have an option for pushing complexity over
from monit to m/monit. 

Thanks (Spasiba balsoje) for the discussion, I'm off to continue work
on m/monit and monit :-)

Paka!


Michael Shigorin <address@hidden> writes:

>>> smtp server is not needed, tiny smtp null mailer only for relay (see
>>> nullmailer, ssmtp, msmtp)
>> I think this is misunderstand - we don't plan to implement SMTP server in 
>> monit.
>
> Well it seems to be miswording. :)
>
>> Monitoring software generaly should be as indepent as possible (Christian 
>> explained reasons).
>
> Yep.  But more importantly, it should work.  More code means more
> problems and development/support/use costs (money, time, nerves...).
>
> Re reasons given:
>
>  > - what if you system doesn't/can't have a smtp server installed?
>
> Then it doesn't need that probably.  If it does, supporting setup with
> e.g. nullmailer is better than handling zillion "what-ifs" for
> supposedly at least a bit knowledgeable to get to monit at all sysadmin.
>
> ICQ counter-question was very reasonable ;-)
>
>  > - what if that certain smtp server fails?
>
> Then it's not the transport.  What if it never comes back up?  Resort
> to scanning internet for open relays? ;-)
>
>  > - what if /usr/lib/sendmail is corrupted?
>
> Disable SMTP transport.
>
>  > - what if the mail server config file is corrupted?
>
> Disable SMTP transport (if corruption is accidental, then the server is
> most likely not to start with it and refuse any client's attempts to
> send mail; if not, then it's likely that we're owned and it's IDS job to
> try and get message out, not monit's)
>
> Look at syslog: it's critical *but* uses UDP nevertheless. (I know NFS
> has suffered badly upon that, but then again -- they tried to implement
> something reliable based on something adherently inreliable and failed,
> even being well-funded)
>
>> Proposed SMTP refactoring serves for more secure SMTP client functionality.
>
> As a humble monit user, I'd argue that users like me *don't* need that
> for the following reasons:
>
> - networked code is more dangerous in general, thus some additional
>    expertise is needed to keep the avreage product level the same;
> - reimplementing the wheel or pulling some (patched) implementation in
>    is *not* an option for reasonable software given that reasonable
>    "wheels" already exist and are usable in component manner.
>
> Let me elaborate a bit.
>
> If you invent yet another SMTP client to rely upon it, you get:
>
> + internal solution, no dependencies, no more complicated code to
>    crash/thrash;
> + it can be more specialized, tiny, lightweight.
>
> It's good.  But:
>
> - if I don't need that and prefer more reliable signalling means, it's
>    more networked code to sit in memory (not listen to the sockets, ok);
> - if the implementation you're basing on has yet unknown security
>    problems, forking/pulling in is *evil*: see e.g. zlib story.
>
>> In the case that all SMTP relays failed, monit will queue messages
>> for later attempt.
>
> Ah, one more thing:
>
> - mail is unreliable by default -- so focusing on it as The Transport
>    is really wrong.  In case of network outages, you're likely better off
>    with GSM modem or serial link based notifier than having a pile of
>    rubble mail when the dust gets down
>
>> This way we will not loose messages in the case of temporary problem (for 
>> example network outage).
>
> But do we need them then?  Isn't logging for that?
>
> The question is whether mail is considered reliable transport, compare
> with UDP vs TCP.
>
> I'd bet that your efforts are unbalanced with the fact that mail (SMTP
> in real world) is *not* reliable for a long time.
>
>>>> - what if that certain smtp server fails?
>>> yes what if "trusted mail hub" fails ? what if smtp session fails
>>> (timeout) ?
>> Monit supports more then one mailhub (smtp relay).
>
> Beautiful.  But more complex code.  Thus more problems with monit
> itself.
>
>> In the case that no relay is available, monit will queue message and try to 
>> deliver it laterr.
>> Again - this is normal for SMTP clients - for examle Mozilla mail
>
> Oh no!  Please don't mix up user software like Mozilla where any bloat
> is acceptable and crashing is no big prob with tools which are leaning
> to the mission-critical area somewhat.
>
>>>> - what if /usr/lib/sendmail is corrupted?
>>> what if monit died with queued message(s) ?
>> In the case of crash you can loose data on any system.
>
> And with more complex watchdog we're closer to crash.  monit is complex
> enough already for me to put it under init(8) directly, disabling
> initscript-based startup.
>
>> We will try to design the queuing method as secure as possible.
>
> You'd really better concentrate on getting things architecturally
> isolated: main process is small, simple code that does its job.  If
> there are any web monitoring/config thingies (which are OK when
> everything's right, but can step in the way when it isn't -- and that
> time is exactly when it *must* work it's best!) -- said thingies are
> separate and can die peacefully by themselves.
>
> Bringing in databases, web/smtp servers/clients must not hurt the main
> virtue of monit -- watching processes and helping them to move on.
>
>> Client code will not mean extra security problem.
>
> Client exploits are a class on their own.  I understand that trusted
> relay shouldn't be that dangerous, but nobody knows.
>
> ------------------------------------------------------------------
>
> See: we've done some mobile-related platform last year.  Some similar
> problems regarding transports involved (SMPP over different wrappers,
> specialized APIs, etc) were predicted, additionally appeared, and solved
> with differing success.
>
> The most obvious mistake in such projects is *counting* on some
> transport to be reliable, and/or making the system single-moduled enough
> to spiral down with all the things when one thing (considered reliable)
> suddenly breaks.
>
> What if your queue takes more resources than available until network is
> available again?  Yes, more complicated queue manager is doable, but
> that's an area of research on its own (we've looked into that, believe
> me) -- and still it's NOT monit's main business doing queues, so it's
> NOT worth main effort.
>
> Even making it not to segfault when parsing bad config would be much
> better investment.  It's different issue (startup-time, not runtime) but
> it is.
>
> Or are you supplied with free time much better than we were with paid
> one in *quite* funded project? ;-)
>
> PS: I'm sorry if the wording is a bit too harsh but bloating code which
> has enough segfaults in the latest production version seems unreasonable
> to me.  Hope that v.4 will be beautiful and reliable -- and I already
> have some plans on integrating support for it in the distribution we
> with Igor help maintaining -- ALT Linux -- and in our server products
> (we're FOSS consulting and development firm in Ukraine), but it would be
> too bad if reimplementing monit itself would be less pain than living
> with too much onboard reimplementations in it.
>
> Hope this helps; if it doesn't make it to the list, could you forward?
> If any replies are on the list, please drop me a Cc:.
>
> Yours,
>
> -- 
> Michael Shigorin
> EMT.Com.UA
>
>
>
> _______________________________________________
> monit-dev mailing list
> address@hidden
> http://mail.nongnu.org/mailman/listinfo/monit-dev
>

-- 
Jan-Henrik Haukeland