Re: [Savannah-hackers-public] vc load

[Bob Proulx]

Karl Berry wrote:
> Apparently loggerhead is started once by rc and lives until the system
> reboots (251 days and counting).  I suppose some kind of balance finally
> got tipped.  Sure looking forward to Bob's reboot program completing.

Each of the VMs has its own set of quirks that I am learning about
them as I go along.  I am just finishing up VM "internal" today.  I am
chatting on IRC with the FSF admins about the Xen bootstrap process
and applying the final cleanup on it now.  (A little scary to be
purging grub and then rebooting but that is the plan today for that
xen domU system.)  That being behind us I will reboot "frontend" and
hit it next.  It is the furthest behind.  But getting into the rhythm
of things now.

But for the initial reboot we might as well reboot "vcs" and
"download" today along with the others.  That will give themn the
initial reboot-before-making-changes since I like being able to
separate old problems that may have been there for a while from new
problems that I may have just introduced.

> There may have been contention with git, since the next three processes
> ...
> Don't know what to make of that.  The git commands all went back to
> inetd but it wasn't obvious which project they were associated with, if
> any.  They completed reasonably quickly (a few minutes) after loggerhead
> was killed, but still.

With a load of 38 they were probably just so slow in running that they
were just still running.  I don't think there would have been any real
contention other than general machine resources.  In particular I have
found that VM I/O is never as fast as native I/O and therefore
anything that really hits the disk hard has a disproportionate effect
of disruption bogging down the machine.

I just installed 'iotop' on all of the machines.  Just run it 'iotop'
and perhaps it will be useful helping to diagnose issues in the
future.  It is like top but shows processes by I/O.  The default
column to sort upon is all IO.  Look for the '>' character to indicate
the sort field.  Use keyboard-left or keyboard-right to change sort
fields.  A useful tool.

> Finally, in /etc/xinetd.d/*, I noticed that all the rlimit_cpu values
> were commented out, as in:
> service git  # or bzr or cvs or whatever
> {
>       disable         = no
>       #rlimit_cpu      = 20
> 
> I've never used rlimit_cpu, but it sure sounds like a good idea.  Did it
> prove problematic in practice?

I have no information here.

Bob