Re: [savannah-help-public] Web interface doesn't work at 4am

[Bob Proulx]

Richard Stallman wrote:
> The web interface to Savannah is failing now (4am Boston time) due to
> timeouts.  I'm told this happens every day at this time.  I guess
> Savannah is doing some sort of lengthy cron job that hogs the machine
> so much the web interface times out.
> 
> Access using cvs is also slow, but it does work.

Thank you for the report.  I know the site uses Nagios monitoring.
The problem hasn't been bad enough to trip the monitoring to report a
problem.  I haven't seen any automated failure reports.  That is
probably why it has continued unnoticed and unfixed.

> I suppose this is the best time to do the cron job.

That is very likely the "daily" cron task that runs all of the
housekeeping tasks.  It runs a directory of scripts one after the
other until they have all completed.  It is sequential so that the
machine isn't swamped with an impulse spike of many concurrent tasks.

At 4am most of the US folks would not notice the problem.  (Because
our machines are also running housekeeping tasks at that time.  And so
I avoid trying to overload my machines during that time period too.)

But Savannah is a VM and the underlying host also has daily cron tasks
to run for housekeeping.  It is possible that the problem isn't on the
Savannah VM itself but on the underyling host that is hosting it.
That would trickle down and cause the VM to be slow even though
nothing on the VM is a direct effect of it.

> Is there a way to make it hog the machine less?
> 
> Limit what fraction of the memory it can have?
> Stop it for 10 seconds after 20 seconds of running?

I will sign up to look into the problem.  The first step will be to
understand what it is on the system that is causing it.  After we
understand the root cause then the solution will likely appear from
it.  And improving the monitoring so that we can be notified when it
appears as a problem.

Bob