Re: cp: improve fallback when fiemap is not supported

[Pádraig Brady]

On 01/12/17 03:43, Dominique Martinet wrote:
> Hi,
> 
> A bit of background first: we use LUSTRE at CEA, and unfortunately it
> does not support fiemap well enough (working on fixing that as well, but
> this will likely take some time)
> We also have many, many sparse files with no data. These are empty
> shells of real files that can be read from tapes transparently on access
> through HSM (hierarchical storage management), but as far as admins as
> concerned we just want to manipulate the empty data.
> 
> For these files, tar has a heuristic that does a stat() and if st_blocks
> is 0 it means there is no extent, and does not read any data.
> I'd like to add the same heuristic to cp if fiemap failed.
> 
> 
> Here's a suggested patch that does this. I could not find any rule
> regarding code IP but feel free to consider this public domain or
> whatever is best for coreutils. (I understand it's not useable as is
> anyway, would likely need to place the function somewhere else, or
> at least make it static; possible rename; add comments; etc. This is
> more about testing water.)
> 
> ----------
> diff --git a/src/extent-scan.c b/src/extent-scan.c
> index bbb686a..e5388a7 100644
> --- a/src/extent-scan.c
> +++ b/src/extent-scan.c
> @@ -81,6 +81,20 @@ extent_scan_init (int src_fd, struct extent_scan *scan)
>  # ifndef FS_IOC_FIEMAP
>  #  define FS_IOC_FIEMAP _IOWR ('f', 11, struct fiemap)
>  # endif
> +
> +bool
> +zero_block (struct extent_scan *scan)
> +{
> +  struct stat buf;
> +  int rc;
> +
> +  rc = fstat(scan->fd, &buf);
> +  if (rc != 0)
> +    return false;
> +
> +  return buf.st_blocks == 0;
> +}
> +
>  /* Call ioctl(2) with FS_IOC_FIEMAP (available in linux 2.6.27) to
>     obtain a map of file extents excluding holes.  */
>  extern bool
> @@ -112,7 +126,14 @@ extent_scan_read (struct extent_scan *scan)
>        if (ioctl (scan->fd, FS_IOC_FIEMAP, fiemap) < 0)
>          {
>            if (scan->scan_start == 0)
> -            scan->initial_scan_failed = true;
> +            {
> +              if (zero_block(scan))
> +                {
> +                  scan->hit_final_extent = true;
> +                  return true;
> +                }
> +              scan->initial_scan_failed = true;
> +            }
>            return false;
>          }
>  
> ------------
> 
> Thank you,
> 


Does upstream tar have this heuristic?
We're looking at moving away from fiemap to using SEEK_HOLE/SEEK_DATA.
Are those supported on your file system?
Would they be applicable in this case?

thanks
Pádraig