Re: [bug #17877] Invalid "No such file or directory" error on filesystem without stable inode numbers

[Jim Meyering]

Eric Blake <address@hidden> wrote:
...
>>   <http://savannah.gnu.org/bugs/?17877>
>
> For the gnulib crowd, a summary of the above link is that find 4.3.0,
> which uses gnulib's fts, is sometimes reporting "No such file or
> directory" in the middle of traversal on non-POSIX file systems such as
> smbfs, cifs, sshfs, or fat, where the combination of the file system
> representation on disk and/or the remote server and caching issues makes
> it possible for a file's inode to change over time; or even for inodes to
> be reused in a short period of time on different files.
>
> Indeed, if the SAME file has two different inodes over time, that file
> system violates POSIX, but such systems do exist, and wreak havoc in
> fts()'s algorithm to traverse directories, because fts is not maintaining
> an open fd to each directory that it is in, but closes and then reopens
> the directories as it goes.  Yet, keeping fds open is expensive - on a
> pathologically deep hierarchy, there are systems that do not have enough
> fds available to traverse the entire hierachy with an open fd for every
> intermediate path, so you have to close and reopen directories as the
> algorithm progresses if you are to visit everything.
>
> On the opposite side of things, I don't know if fts() is or can be made
> robust to two distinct files being given the same inode due to an unlink
> and new file creation occuring during traversal, although I seem to recall
> an attempt in the past to make rm (which uses fts) more robust along these
> lines.

IMHO, it was more than an "attempt" to fix the bug in GNU rm :-)
Here's the thread:
  http://article.gmane.org/gmane.comp.gnu.core-utils.bugs/6614
Note that rm does not use fts.

But both rm and fts do use the same underlying cycle-detection module.
The sole place where dev/inode reuse matters is in cycle detection.
One difference is that rm uses the CYCLE_CHECK_REFLECT_CHDIR_UP
macro (from cycle-check.h) whenever it does a virtual chdir("..").
However, even that macro is necessary only when fts is using
its default (lazy, O(1)) cycle detection method.

I'll prepare a patch that makes fts use that macro in lazy
cycle-detection mode -- but not today.

For find, the easiest approach to avoiding failure due to a false-positive
cycle detection may be simply to add the FTS_TIGHT_CYCLE_CHECK option
to the flags used when calling fts_open.  That makes fts maintain a
small hash table of dev/inode pairs corresponding to the set of "active"
directories.  Since the directory dev/inode pairs in question cannot be
reused while a directory is active, in that mode, fts is not subject to
the problems I think you're referring to.