[Fab-user] Re: Server downtime

[Jeff Forcier]

Update: while I had to bounce Apache a number of times during the
afternoon/evening as a stopgap solution, I think I've managed to block
the bots from crawling the problem areas for now.

There will likely be more downtime in the future as I clean up the
existing solution a bit so it's a better long-term one (or if the
problem recurs.) However, with luck it will be minimal.

Thanks,
Jeff

On Fri, Nov 13, 2009 at 4:15 PM, Jeff Forcier <address@hidden> wrote:
> Hi all,
>
> Just a note in case anyone has been trying to access the server, it
> looks like some search engine bot is crawling the site and causing it
> to overheat, so to speak (going OOM in a big way.) Not 100% positive
> that's the cause, but _some_ traffic pattern in the past few hours is
> causing the Redmine instance to gobble RAM where it was fine for the
> last couple of months.
>
> Thus, I've had to restart or shut down Apache a couple of times
> recently, meaning *.fabfile.org has been intermittently down for the
> count.
>
> I'll be re-examining my Passenger config to see why it's OOMing
> instead of simply refusing requests (probably need to rebalance
> #-workers / allowed max memory / etc sorts of settings) in addition to
> probably blacklisting this web crawler in my robots.txt. Hopefully
> within the next hour or two, tops.
>
> Thanks for your patience :)
>
> -Jeff
>
> --
> Jeff Forcier
> Unix sysadmin; Python/Ruby developer
> http://bitprophet.org
>



-- 
Jeff Forcier
Unix sysadmin; Python/Ruby developer
http://bitprophet.org