Re: [Bug-tar] Sparse file performance and suggestions

[Joerg Schilling]

Eric Blake <address@hidden> wrote:

> > I asume that you are testing on a OS that does not implement 
> > SEEK_HOLE/SEEK_DATA, as star is even faster that GNU tar in case that the 
> > OS 
> > helps to retrieve the sparse file info.
>
> Solaris has SEEK_HOLE, although the new coreutils cp code for using
> efficient sparse traversal has not yet been ported to use it.  It's on
> the list of things to implement (and has been for more than 6 months).

Jeff Bonwick and I defined the SEEK_HOLE/SEEK_DATA interface in September 2004, 
and I implemented support for it in star in May 2005 - a few weeks after the 
implementation in Solaris was ready. At that time, there was nothing similar 
and I was in hope that other OS just reimplement this useful idea.

> Linux has ioctl(FIEMAP), which is just as efficient as Solaris'
> SEEK_HOLE, and coreutils 8.10 is the first GNU program to use it.  We
> are planning on moving coreutils' sparse traversal into gnulib (where it
> can be used by tar), as well as enhancing it to recognize Solaris'
> SEEK_HOLE, at which point, GNU tar should indeed be faster at
> recognizing already sparse files, and at which point you will indeed
> want a new tuning option that controls whether tar faithfully copies the
> existing holes of the source files (faster, but overlooks non-sparse
> blocks of 0s that could have been made sparse), vs. finding all runs of
> 0s (slower, done by current behavior, can result in a sparser copy than
> the original, and can still be made somewhat faster by skipping known
> holes).

The main problem with FIEMAP is that I could not find a useful description on 
it's behavior and that it seems to be higly compley without need. 

A problem with gnu coreutils is that it uses a restrictive license and for 
that reason, Tim Kientzle and I cannot use the code for our implementations. 
Do you have a short piece of example code for FIEMAP that returns hole/data 
pairs and that allows to detect sparse files?

http://www.fokus.fraunhofer.de/usr/schilling    ftp://ftp.berlios.de/pub/schily