[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: fts: do not exhaust memory when processing million-entry directory

From: James Youngman
Subject: Re: fts: do not exhaust memory when processing million-entry directory
Date: Mon, 22 Aug 2011 02:41:08 +0100

2011/8/19 Pádraig Brady <address@hidden>:
> file descriptors a usually more constrained so I'd
> lean towards using more memory.

Of course for fts, it's possible to take an adaptive strategy also.
If at any time, fts fails to open a directory with ENFILE, it can
check the parent directories; examining only the directories which are
still open, it can pick the one with the smallest filesize, finish
reading the directory entries of the ancestor and therefore call
closedir on it, and then resume its attempt to open a directory now
tha ta file descriptor has been freed.   Of course this would add
significant code complexity and deal only with a case that will be
quite rare.   So I'm not suggesting we implement it, just suggesting
that we need not be forced into accepting any specific limit here.


This email is intended solely for the use of its addressee, sender,
and any readers of a mailing list archive in which it happens to
appear.   If you have received this email in error, please say or type
three times, "I believe in the utility of email disclaimers," and then
reply to the author correcting any spellings (and, optionally, any
incorrect spellings), accompanying these with humorous jests about the
author's parentage.   If you are not the addressee, you are
nevertheless permitted to both copy and forward this email since
without such permissions email systems are unable to transmit email to
anybody, intended recipient or not.  To those still reading by this
point, the author would like to apologise for being unable to maintain
a consistent level of humour throughout this disclaimer.  Contents may
settle during transit.  Do not feed the animals.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]