Re: Almost-infinite loop behavior in NSFileManager

[Lobron, David]

I just realized that this is likely being caused by a change to the target file 
(made by a different process).  

Would it make sense for NSFileManager::copyPath to periodically recompute the 
file size, and to throw an exception if it changes?  The current code has a 
race condition between the initial file size calculation and the read/write 
loop.  If the file gets smaller, it will hit EOF before the loop hits its exit 
condition, and the loop will never finish.

--David

> On Oct 6, 2016, at 12:02 PM, Lobron, David <dlobron@akamai.com> wrote:
> 
> Hi All,
> 
> I have an application that calls NSFileManager's copyPath:toPath:handler: 
> method on a 40MB file.  I'm finding that occasionally, the copy takes an 
> extremely long time, up to a few days.
> 
> I attached the process and found that it's bouncing between read() and 
> write() system calls in NSFileManager.m.  Stack traces show either this:
> 
> #0  0x00007efe820f389d in read () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x00007efe83014d07 in read (__nbytes=8096, __buf=0x7ffc7a12cb00, __fd=14) 
> at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
> #2  -[NSFileManager(PrivateMethods) _copyFile:toFile:handler:] 
> (self=0x1fb45c0, _cmd=<optimized out>, source=0x2a07810, 
> destination=0x220bd730, handler=0x0)
>    at NSFileManager.m:2748
> #3  0x00007efe830122eb in -[NSFileManager copyPath:toPath:handler:] 
> (self=0x1fb45c0, _cmd=<optimized out>, source=0x2a07810, 
> destination=0x220bd730, handler=0x0)
>    at NSFileManager.m:1109
> 
> or this:
> 
> #0  0x00007efe820f38fd in write () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x00007efe83014ce1 in -[NSFileManager(PrivateMethods) 
> _copyFile:toFile:handler:] (self=0x1fb45c0, _cmd=<optimized out>, 
> source=0x2a07810, destination=0x220bd730, 
>    handler=0x0) at NSFileManager.m:2761
> #2  0x00007efe830122eb in -[NSFileManager copyPath:toPath:handler:] 
> (self=0x1fb45c0, _cmd=<optimized out>, source=0x2a07810, 
> destination=0x220bd730, handler=0x0)
>    at NSFileManager.m:1109
> 
> I checked the process with strace, and confirmed that it's bouncing between 
> read and write, both of which are returning 0:
> 
> read(14, "", 8096)                      = 0
> write(15, "", 0)                        = 0
> read(14, "", 8096)                      = 0
> write(15, "", 0)                        = 0
> read(14, "", 8096)                      = 0
> write(15, "", 0)                        = 0
> ...etc.
> 
> The code in question is:
> 
>  /* Read bufsize bytes from source file and write them into the destination   
>                                                                               
>             
>     file. In case of errors call the handler and abort the operation. */
>  for (i = 0; i < fileSize; i += rbytes)
>    {
>      rbytes = read (sourceFd, buffer, bufsize);
>      if (rbytes < 0)
>        {
>          close (sourceFd);
>          close (destFd);
> 
>          return [self _proceedAccordingToHandler: handler
>                                         forError: @"cannot read from file"
>                                           inPath: source
>                                         fromPath: source
>                                           toPath: destination];
>        }
> 
>      wbytes = write (destFd, buffer, rbytes);
>      if (wbytes != rbytes)
>        {
>          close (sourceFd);
>          close (destFd);
> 
>          return [self _proceedAccordingToHandler: handler
>                                         forError: @"cannot write to file"
>                                           inPath: destination
>                                         fromPath: source
>                                           toPath: destination];
>        }
>    }
> 
> I printed out the values of i, fileSize, bufSize and rbytes:
> 
> (gdb) print i
> $1 = 44102943
> (gdb) print fileSize
> $2 = 44113575
> (gdb) print bufsize
> $6 = 8096
> 
> Most interesting was the value of rbytes:
> 
> (gdb) print rbytes
> $4 = 0
> 
> Later:
> 
> (gdb) print rbytes
> $5 = <optimized out>
> 
> Since read is returning 0, the loop counter is not incremented.  The read(2) 
> manpage says: "On  success, the number of bytes read is returned (zero 
> indicates end of file), and the file position is advanced by this number.  It 
> is not an error if this number is smaller than the number of bytes requested; 
> this may happen for example because fewer bytes are actually available right 
> now (maybe because we were close  to  end-of-file, or because we are reading 
> from a pipe, or from a terminal), or because read() was interrupted by a 
> signal."
> 
> It seems like read is hitting EOF before we've read fileSize bytes.  Or maybe 
> one of the other conditions that causes read to return less than the number 
> of bytes read is being hit.  Has anyone seen this before? 
> 
> Thanks,
> 
> David
> 
> 
>