[bug #24140] Painfully slow find(1) in list-permission-only AFS paths

[Daniel Richard G.]

URL:
  <http://savannah.gnu.org/bugs/?24140>

                 Summary: Painfully slow find(1) in list-permission-only AFS
paths
                 Project: findutils
            Submitted by: iskunk
            Submitted on: Tue 26 Aug 2008 07:03:01 AM GMT
                Category: find
                Severity: 3 - Normal
              Item Group: None
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 4.4.0
           Fixed Release: None

    _______________________________________________________

Details:

I have a script intended to generate a file listing of several users' home
directories, stored in AFS. The listing should be fairly bare-bones: no file
attributes of any kind, not even a distinction between regular files vs.
symlinks vs. other file types, just names and the directories containing
them.

Consistent with that, the script runs under a restricted permission set where
it can only list directories. Where an AFS admin would have all "rlidwka" bits
set, this script would only get "l" (list) access.

find(1) takes a very long time scanning directory trees under these
conditions. Here is what "strace -r -p <find-pid>" says is going on:

...
     3.016894 lstat("Makefile", 0x7fff950fb590) = -1 EACCES (Permission
denied)
     3.001573 newfstatat(AT_FDCWD, "abuzz_r3_linguistics.tex", 0x664278,
AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied)
     3.019501 lstat("abuzz_r3_linguistics.tex", 0x7fff950fb590) = -1 EACCES
(Permission denied)
     3.001665 newfstatat(AT_FDCWD, "abuzz_r3.tex", 0x664398,
AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied)
     3.020538 lstat("abuzz_r3.tex", 0x7fff950fb590) = -1 EACCES (Permission
denied)
     3.001657 newfstatat(AT_FDCWD, "abuzz_r3_status.tex", 0x664b48,
AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied)
     3.023643 lstat("abuzz_r3_status.tex", 0x7fff950fb590) = -1 EACCES
(Permission denied)
     3.001544 newfstatat(AT_FDCWD, "abuzz_r3_misc.tex", 0x664c68,
AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied)
     3.019582 lstat("abuzz_r3_misc.tex", 0x7fff950fb590) = -1 EACCES
(Permission denied)

Each of those failed stat calls is taking over three seconds! Per file! And
this is with 4.4.0 compiled with d_type and leaf optimization enabled.

find(1) should not be stat()ing those files to begin with. I didn't give it a
-type predicate, and it can already tell which ones are directories via
d_type, so why go to the trouble?

As a potential (new) optimization, I vaguely recall that when doing readdir()
for an AFS file with only "l" access, d_type would have an value corresponding
to "unknown." In which case, find(1) can assume that a stat() is pointless
because it would invariably fail. The thing to remember is that, with "l"
access only, AFS will allow you to see the name, but will tenaciously withhold
all other info, even file type (with the exception of directories). The fix
is, effectively, to handle this situation in a more graceful way.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?24140>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/