Re: Almost-infinite loop behavior in NSFileManager

[Lobron, David]

Yes, thanks- I saw his change and it looks very good.

--David

> On Oct 11, 2016, at 2:23 PM, Ivan Vučica <ivan@vucica.net> wrote:
> 
> In case you didn’t see it: rfm has recently submitted a change which 
> repeatedly compares modification date.
> On 6 October 2016 at 18:11:42, Lobron, David (dlobron@akamai.com) wrote:
> 
>> I just realized that this is likely being caused by a change to the target 
>> file (made by a different process).  
>> 
>> Would it make sense for NSFileManager::copyPath to periodically recompute 
>> the file size, and to throw an exception if it changes? The current code has 
>> a race condition between the initial file size calculation and the 
>> read/write loop. If the file gets smaller, it will hit EOF before the loop 
>> hits its exit condition, and the loop will never finish. 
>> 
>> --David 
>> 
>> > On Oct 6, 2016, at 12:02 PM, Lobron, David <dlobron@akamai.com> wrote: 
>> >  
>> > Hi All, 
>> >  
>> > I have an application that calls NSFileManager's copyPath:toPath:handler: 
>> > method on a 40MB file. I'm finding that occasionally, the copy takes an 
>> > extremely long time, up to a few days. 
>> >  
>> > I attached the process and found that it's bouncing between read() and 
>> > write() system calls in NSFileManager.m. Stack traces show either this: 
>> >  
>> > #0 0x00007efe820f389d in read () from /lib/x86_64-linux-gnu/libc.so.6 
>> > #1 0x00007efe83014d07 in read (__nbytes=8096, __buf=0x7ffc7a12cb00, 
>> > __fd=14) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44 
>> > #2 -[NSFileManager(PrivateMethods) _copyFile:toFile:handler:] 
>> > (self=0x1fb45c0, _cmd=<optimized out>, source=0x2a07810, 
>> > destination=0x220bd730, handler=0x0) 
>> > at NSFileManager.m:2748 
>> > #3 0x00007efe830122eb in -[NSFileManager copyPath:toPath:handler:] 
>> > (self=0x1fb45c0, _cmd=<optimized out>, source=0x2a07810, 
>> > destination=0x220bd730, handler=0x0) 
>> > at NSFileManager.m:1109 
>> >  
>> > or this: 
>> >  
>> > #0 0x00007efe820f38fd in write () from /lib/x86_64-linux-gnu/libc.so.6 
>> > #1 0x00007efe83014ce1 in -[NSFileManager(PrivateMethods) 
>> > _copyFile:toFile:handler:] (self=0x1fb45c0, _cmd=<optimized out>, 
>> > source=0x2a07810, destination=0x220bd730,  
>> > handler=0x0) at NSFileManager.m:2761 
>> > #2 0x00007efe830122eb in -[NSFileManager copyPath:toPath:handler:] 
>> > (self=0x1fb45c0, _cmd=<optimized out>, source=0x2a07810, 
>> > destination=0x220bd730, handler=0x0) 
>> > at NSFileManager.m:1109 
>> >  
>> > I checked the process with strace, and confirmed that it's bouncing 
>> > between read and write, both of which are returning 0: 
>> >  
>> > read(14, "", 8096) = 0 
>> > write(15, "", 0) = 0 
>> > read(14, "", 8096) = 0 
>> > write(15, "", 0) = 0 
>> > read(14, "", 8096) = 0 
>> > write(15, "", 0) = 0 
>> > ...etc. 
>> >  
>> > The code in question is: 
>> >  
>> > /* Read bufsize bytes from source file and write them into the destination 
>> >  
>> > file. In case of errors call the handler and abort the operation. */ 
>> > for (i = 0; i < fileSize; i += rbytes) 
>> > { 
>> > rbytes = read (sourceFd, buffer, bufsize); 
>> > if (rbytes < 0) 
>> > { 
>> > close (sourceFd); 
>> > close (destFd); 
>> >  
>> > return [self _proceedAccordingToHandler: handler 
>> > forError: @"cannot read from file" 
>> > inPath: source 
>> > fromPath: source 
>> > toPath: destination]; 
>> > } 
>> >  
>> > wbytes = write (destFd, buffer, rbytes); 
>> > if (wbytes != rbytes) 
>> > { 
>> > close (sourceFd); 
>> > close (destFd); 
>> >  
>> > return [self _proceedAccordingToHandler: handler 
>> > forError: @"cannot write to file" 
>> > inPath: destination 
>> > fromPath: source 
>> > toPath: destination]; 
>> > } 
>> > } 
>> >  
>> > I printed out the values of i, fileSize, bufSize and rbytes: 
>> >  
>> > (gdb) print i 
>> > $1 = 44102943 
>> > (gdb) print fileSize 
>> > $2 = 44113575 
>> > (gdb) print bufsize 
>> > $6 = 8096 
>> >  
>> > Most interesting was the value of rbytes: 
>> >  
>> > (gdb) print rbytes 
>> > $4 = 0 
>> >  
>> > Later: 
>> >  
>> > (gdb) print rbytes 
>> > $5 = <optimized out> 
>> >  
>> > Since read is returning 0, the loop counter is not incremented. The 
>> > read(2) manpage says: "On success, the number of bytes read is returned 
>> > (zero indicates end of file), and the file position is advanced by this 
>> > number. It is not an error if this number is smaller than the number of 
>> > bytes requested; this may happen for example because fewer bytes are 
>> > actually available right now (maybe because we were close to end-of-file, 
>> > or because we are reading from a pipe, or from a terminal), or because 
>> > read() was interrupted by a signal." 
>> >  
>> > It seems like read is hitting EOF before we've read fileSize bytes. Or 
>> > maybe one of the other conditions that causes read to return less than the 
>> > number of bytes read is being hit. Has anyone seen this before?  
>> >  
>> > Thanks, 
>> >  
>> > David 
>> >  
>> >  
>> >  
>> 
>> 
>> _______________________________________________ 
>> Discuss-gnustep mailing list 
>> Discuss-gnustep@gnu.org 
>> https://lists.gnu.org/mailman/listinfo/discuss-gnustep