[bug #30868] Take advantage of dirent.d_type to save a stat()

[Nick Welch]

Follow-up Comment #1, bug #30868 (project grep):

I have attached a proof of concept patch that implements this.  I'm sure it
breaks some coding conventions and could use some adjustment, but it does work
and passes the test suite.

Watching the action through strace, it now avoids calling stat() on excluded
files at all.  (It's worth noting that this optimization only comes into play
when excluding/including files/dirs.)

The conditions under which this is a significant speedup appear to be fairly
rare.  The cost of stat()ing files seems be a low percentage of the overall
runtime.  Even when reading through file contents is completely avoided by
doing something like --include '*.asdfasdfasdf', the speedup is only around
10% for me (searching through all of /usr recursively).  Under normal
conditions the speed improvement seems to be lost in noise.

However, these measurements could vary wildly between OS, filesystem,
hardware, etc., for all I know.  I tested the latest grep from git, on Ubuntu
10.04, on a mechanical hard drive running ext4.  I also made sure to drop all
disk caches before each test by echoing 3 to /proc/sys/vm/drop_caches.

I'm also confused about how the savedir code is maintained.  The version in
gnulib has the same interface but is significantly different from the version
in grep.  The gnulib version doesn't even attempt to check file type.

(file #21311)
    _______________________________________________________

Additional Item Attachment:

File name: 0001-Check-dirent.d_type-to-avoid-many-stat-s.patch Size:1 KB


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?30868>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/