bug#6131: [PATCH]: fiemap support for efficient sparse file copy

[jeff.liu]

Sunil Mushran wrote:
> On 06/10/2010 04:47 PM, Paul Eggert wrote:
>> On 06/09/2010 11:56 PM, jeff.liu wrote:
>>   
>>> Yeah, I just realized that the behaviour I observed is caused by the
>>> delay allocation mechanism of
>>> the particular FS.
>>>      
>> If the file system is using delayed allocation, then can
>> the fiemap ioctl tell us that a file contains a hole (because nothing
>> has been
>> allocated there), but read() would tell us that the file contains
>> nonzero data at the same location
>> (because it's sitting in a buffer somewhere)?  If so, we'd need to do
>> something like invoke
>> fdatasync() on the file before issuing the fiemap ioctl, to force
>> allocation; or perhaps
>> there's another ioctl that will do the allocation without having to
>> actually do a sync.
>>    
> 
> I guess we'll have to use FIEMAP_FLAG_SYNC.
Hi Sunil,

Thanks for the comments.
So we can ensure the source file synced before mapping in this way.

Hi Jim and Paul,

How about the tiny patch below?

>From d6d619a169ff68a9a310a69d8089b9fbf83b5f91 Mon Sep 17 00:00:00 2001
From: Jie Liu <address@hidden>
Date: Fri, 11 Jun 2010 16:29:02 +0800
Subject: [PATCH 1/1] copy.c: add FIEMAP_FLAG_SYNC to fiemap ioctl

* src/copy.c (fiemap_copy): Force kernel to sync the source
file before mapping.

Signed-off-by: Jie Liu <address@hidden>
---
 src/copy.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index f149be4..f48c74d 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -191,6 +191,7 @@ fiemap_copy (int src_fd, int dest_fd, size_t buf_size,
   do
     {
       fiemap->fm_length = FIEMAP_MAX_OFFSET;
+      fiemap->fm_flags = FIEMAP_FLAG_SYNC;
       fiemap->fm_extent_count = count;

       /* When ioctl(2) fails, fall back to the normal copy only if it
-- 
1.5.4.3



Thanks,
-Jeff

> 
>> There's also the issue of copying from a file at the same time that
>> some other process
>> is writing to it, but that is allowed to produce ill-defined
>> behavior.  I'm more worried
>> about the case where some other process writes to the source file just
>> before 'cp' starts.
>>    
> 
> cp's behavior with active files is undefined. But we know it reads from
> offset 0 to MAX. With fiemap it will continue to do the same with the
> exception that it will skip reads (and thus writes) depending on the extent
> map it gets at the very beginning.
> 
>> (Sorry, I haven't had time yet to dive into the proposed change; I'm
>> still trying to understand
>> the environment.)
>>
>> One other thing: Solaris 10 supports lseek with the SEEK_HOLE and
>> SEEK_DATA options, which
>> are easier to use and which (as far as I can tell from the manual)
>> shouldn't require anything
>> fdatasync-ish.  Any objection if I propose support for that too?  It
>> is supposed to work
>> with ZFS, something I can test here.
>>    
> 
> There is no plan to implement SEEK_HOLE/SEEK_DATA in the kernel.
> At most glibc will use fiemap to extend lseek(). BTW, SEEK_HOLE/DATA
> also have the same problem with active files.
> 
> ccing linux-ext4.
> 
> 
> 


-- 
With Windows 7, Microsoft is asserting legal control over your computer and is 
using this power to
abuse computer users.