[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: stat() order performance issues

From: Philip Rowlands
Subject: Re: stat() order performance issues
Date: Fri, 26 Jan 2007 02:54:19 +0000 (GMT)

On Thu, 25 Jan 2007, Phillip Susi wrote:

I have noticed that performing commands such as ls ( even with -U ) and du in a Maildir with many thousands of small files takes ages to complete. I have investigated and believe this is due to the order in which the files are stat()ed. I believe that these utilities are simply stat()ing the files in the order that they are returned by readdir(), and this causes a lot of random disk reads to fetch the inodes from disk out of order.

My initial testing indicates that sorting the files into inode order and calling stat() on them in order is around an order of magnitude faster, so I would suggest that utilities be modified to behave this way.

If I shared the same filesystem type, kernel, and I/O subsystem then I would likely be able to reproduce these results, but...

It's not really possible for the kernel to help by re-ordering the requests to disk, as the system calls are issued serially.

Although there may be cases where "stat in inode order" doesn't help, I'm struggling to see how it could be worse than "stat in directory order", especially for high-churn directories. Are there filesystems which re-order directories to optimize access patterns? At a quick glance, I see nothing in POSIX which disallows this.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]