[coreutils] Feature: fiemap_copy for cp/mv

[jeff.liu]

Hello,

This is the second try to add fiemap copy feature to cp/mv.
First off, I'd like to introduce this feature to cp(1) for the review, then 
deal with mv(1) if you
guys like the current implementation.

The bug pointed by joel in the first try was fixed, thanks!

Here is a short declaration for fiemap ioctl(2), this feature was introduced in 
to mainline kernel
in 2.6.27.

FIEMAP supply a straigtforward approach to efficiently map file allocation of a 
file, call fiemap
ioctl(2) againt a file descriptor will returns the allocated extents info, with 
the info, we can
easily determine each extent's logical start offset and its length, that's is 
to say, where and how
many bytes need to copy(depending on the bytes read(2) for each extent).

This patchset is intend to be an optimation for the sparse file copy, it could 
reduce the efforts
of verify holes for current copy_reg() implementation.

For each sparse file copy done, its size was truncated to stat.st_size to 
respect its apparent size
ls(1) showing. To determine if a sparse file was encountered, Here I do a 
stupid check by comparing
the sum of the last extent logical offset and its actual length(not the extent 
length) against the
(struct stat) sbuf.st_size, any criticism or suggestion for this implementation 
were appreciated!

cp --fiemap=[WHEN] is used to tigger the fiemap copy.
cp --fiemap=always, always doing fiemap copy, the copy process will abort if 
the underlaying
filesystem does not support fiemap.
cp --fiemap=auto, try to do fiemap copy, it will fall back to do normal copy 
for any failure(i.e.,
the underlying filesystem does not support fiemap).
cp --fiemap=[WHEN] combine with --fiemap-sync will cause the underlaying 
filesystem sync data file
before map. I'd omit the FIEMAP_XATTR_FLAG in this case, call ioctl(2) with 
this flag with cause the
filesystem do map the extended attributes tree instead, as cp(1) could deal 
with xattr preserve if
it was built with the corresponding library and the --preserve=xattr is 
specified.

I have done some tests, they are shown great performance boost for sparse 
files, for normal file
copy, the performance is pretty much the same thing.

Please check the following for details:

jeff@jeff-laptop:~/opensource_dev/coreutils$ time ./src/cp --fiemap=always 
/ocfs2/sparse_file4
/ocfs2/sparse_file4_fiemap_copy

real    0m0.058s
user    0m0.000s
sys     0m0.048s
jeff@jeff-laptop:~/opensource_dev/coreutils$ time ./src/cp --sparse=always 
/ocfs2/sparse_file4
/ocfs2/sparse_file4_normal_copy

real    0m8.884s
user    0m1.128s
sys     0m6.752s
jeff@jeff-laptop:~/opensource_dev/coreutils$ ls -l /ocfs2/sparse_file4*
-rw-r--r-- 1 jeff jeff 838860800 Mar 29 16:18 /ocfs2/sparse_file4
-rw-r--r-- 1 jeff jeff 838860800 Mar 29 16:18 /ocfs2/sparse_file4_fiemap_copy
-rw-r--r-- 1 jeff jeff 838860800 Mar 29 16:19 /ocfs2/sparse_file4_normal_copy
jeff@jeff-laptop:~/opensource_dev/coreutils$ du -sh /ocfs2/sparse_file4*
352K    /ocfs2/sparse_file4
352K    /ocfs2/sparse_file4_fiemap_copy
768K    /ocfs2/sparse_file4_normal_copy

jeff@jeff-laptop:~/opensource_dev/coreutils$ time ./src/cp --fiemap=always 
/ext4/sparse_file4
/ext4/sparse_file4_fiemap_copy

real    0m0.184s
user    0m0.004s
sys     0m0.048s
jeff@jeff-laptop:~/opensource_dev/coreutils$ time ./src/cp --sparse=always 
/ext4/sparse_file4
/ext4/sparse_file4_normal_copy

real    0m6.536s
user    0m1.332s
sys     0m4.304s

jeff@jeff-laptop:~/opensource_dev/coreutils$ ls -l /ext4/sparse_file4*
-rw-r--r-- 1 jeff jeff 838860800 Apr  3 21:04 /ext4/sparse_file4
-rw-r--r-- 1 jeff jeff 838860800 Apr  3 21:40 /ext4/sparse_file4_fiemap_copy
-rw-r--r-- 1 jeff jeff 838860800 Apr  3 21:40 /ext4/sparse_file4_normal_copy
jeff@jeff-laptop:~/opensource_dev/coreutils$ du -sh /ext4/sparse_file4*
772K    /ext4/sparse_file4
768K    /ext4/sparse_file4_fiemap_copy
768K    /ext4/sparse_file4_normal_copy

non-sparse file:

jeff@jeff-laptop:~/opensource_dev/coreutils$ time ./src/cp --fiemap=always
/ocfs2/linux-2.6.34-rc1.tar.bz2 /ocfs2/linux-2.6.34-rc1.tar.bz2_fiemap_copy

real    0m3.734s
user    0m0.020s
sys     0m3.684s
jeff@jeff-laptop:~/opensource_dev/coreutils$ time ./src/cp 
/ocfs2/linux-2.6.34-rc1.tar.bz2
/ocfs2/linux-2.6.34-rc1.tar.bz2_normal_copy

real    0m3.808
user    0m0.000s
sys     0m3.720s

Any comments are welcomed!


Thanks,
-Jeff