Re: a) HTTP-Port & quitting, b) monit ran amok on my apache

[Martin Pala]

Hi, seems prety messy :(

Can you send output from truss?

Martin

----- Original Message -----
From: "Christian Hopp" <address@hidden>
To: "Monit Mailinglist" <address@hidden>
Sent: Monday, July 01, 2002 11:52 AM
Subject: a) HTTP-Port & quitting, b) monit ran amok on my apache


Hi!

I am using Monit (right now version 2.4.2) with Solaris 2.7 (with most
recent patchclusters) on Sparc (U10/60, E450).  It's compiled with
gcc version 2.95.3 20010315 (release).

Problem a)
----------

Mostly when monit is quitted (SIGQUIT, SIGTERM and SIGKILL) there
happens something really strange... monit is quited but many of my
servers are reported to listen to monit's HTTP port (here 2812).
Thus, monit can't restart with HTTP support, because the port is
already occupied (by the way, monit reports itself to be started with
HTTP-support on the command line but it shows on syslog that it
doesn't).  Look at the following lsof report:

iei1-root: /var/adm 50 > lsof | grep 2812
python2.1 11959     root ... TCP *:2812 (LISTEN)
python2.1 11960    httpd ... TCP *:2812 (LISTEN)
sh        12066     root ... TCP *:2812 (LISTEN)
custom_lo 12068     root ... TCP *:2812 (LISTEN)
sh        12081     root ... TCP *:2812 (LISTEN)
custom_lo 12082     root ... TCP *:2812 (LISTEN)
httpd     12083     root ... TCP *:2812 (LISTEN)
sh        12094     root ... TCP *:2812 (LISTEN)
custom_lo 12095     root ... TCP *:2812 (LISTEN)
sh        12100     root ... TCP *:2812 (LISTEN)
custom_lo 12103     root ... TCP *:2812 (LISTEN)
httpd     12104    httpd ... TCP *:2812 (LISTEN)
(...)
httpd     12180    httpd ... TCP *:2812 (LISTEN)

I have never seen a sh listening on a port.

Hmmm... I think this weekend it got even worse... now all those above
tasks AND monit are listening to 2812 (without quitting monit).  And
monit it already running since the 28th...

iei1-root: /var/adm 59 > ps -aef | grep monit
    root   509     1  0   Jun 28 ?       21:10 /opt/bin/monit
            -c /opt/etc/monitrc

and now this...

iei1-root: /export/dist 37 > lsof | grep 2812
monit       509     root ... TCP *:2812 (LISTEN)
python2.1  5626     root ... TCP *:2812 (LISTEN)
python2.1  5627    httpd ... TCP *:2812 (LISTEN)
sh         5775     root ... TCP *:2812 (LISTEN)
httpd      5778     root ... TCP *:2812 (LISTEN)
custom_lo  5780     root ... TCP *:2812 (LISTEN)
sh         5788     root ... TCP *:2812 (LISTEN)
custom_lo  5790     root ... TCP *:2812 (LISTEN)
sh         5791     root ... TCP *:2812 (LISTEN)
custom_lo  5792     root ... TCP *:2812 (LISTEN)
httpd     16984    httpd ... TCP *:2812 (LISTEN)
(...)
httpd     17027    httpd ... TCP *:2812 (LISTEN)

So now I restarted apache (monit restart apache) and hallelujah... the
new instance listens still to 2812. Let's try zope (the two python2.1
instances)... the same... the new one still listens to 2812.  The only
help is to stop apache, zope and monit and afterwards start them one
after an other.  That looks better...

iei1-root: /root 99 > lsof | grep 2812
monit     18903     root ... TCP *:2812 (LISTEN)


By now, the only programs which are acting like this with monit are
Apache/1.3.26 (and it's helpers) and Zope/2.4.3 (our application
server written in python).  Thus I have seen this phenomenon just on
our server (that's why it bugs me!) and not on our Sparc-Workstations.


Problem b)
----------

Furthermore monit ran amok on my apache this Sunday.  Apache is
stopped (with 'monit stop apache') and started (with 'monit start
apache') in the night for minute (logfiles and so on).  After the run
on Sunday morning monit restarted my apache because apache returned
503.  Look at this syslog extract.

Jun 30 09:20:56 iei1 monit[509]:
 HTTP error: HTTP/1.1 503 Service Unavailable
Jun 30 09:20:56 iei1 monit[509]:
 'apache', test with protocol [HTTP] failed at localhost:80.
Jun 30 09:20:56 iei1 monit[509]:
 Trying to restart 'apache'
Jun 30 09:20:56 iei1 monit[509]:
 Stop: (apache) /etc/init.d/apache stop
Jun 30 09:21:06 iei1 monit[509]:
 Start: (apache) /etc/init.d/apache start
Jun 30 09:23:10 iei1 monit[509]:
 HTTP error: HTTP/1.1 503 Service Unavailable
Jun 30 09:23:10 iei1 monit[509]:
 'apache', test with protocol [HTTP] failed at localhost:80.
Jun 30 09:23:10 iei1 monit[509]:
 Trying to restart 'apache'
Jun 30 09:23:10 iei1 monit[509]:
 Stop: (apache) /etc/init.d/apache stop
Jun 30 09:23:20 iei1 monit[509]:
 Start: (apache) /etc/init.d/apache start

The 503 (Service Unavailable) was also reported on access_log.

But the rest of apache worked fine (normal accesses have taken place).
In the morning of Monday apache was restarted on plan and monit
behaved normally again.  Might it be connected to Problem a)?


I need help...!!!

Bye,

C.Hopp

--
Christian Hopp                                email:
address@hidden
Institut für Elektrische Informationstechnik             fon:
+49-5323-72-2113
Technische Universität Clausthal                         fax:
+49-5323-72-3197
  pgpkey: https://www.iei.tu-clausthal.de/pgp-keys/chopp.key.asc
(2001-11-22)


--
To unsubscribe:
http://mail.freesoftware.fsf.org/mailman/listinfo/monit-general