Re: [Qemu-devel] [PATCH v3 1/6] cutils: extract buffer_is_zero() from qemu-img.c

[Stefan Hajnoczi]

On Wed, Dec 21, 2011 at 09:43:55AM -0700, Eric Blake wrote:
> On 12/21/2011 09:00 AM, Stefan Hajnoczi wrote:
> > The qemu-img.c:is_not_zero() function checks if a buffer contains all
> > zeroes.  This function will come in handy for zero-detection in the
> > block layer, so clean it up and move it to cutils.c.
> > 
> > Note that the function now returns true if the buffer is all zeroes.
> > This avoids the double-negatives (i.e. !is_not_zero()) that the old
> > function can cause in callers.
> 
> Are there plans to improve the efficiency of buffer_is_zero to take
> advantage of metadata about sparseness?
> 
> That is, there are cases where we can use metadata to prove a region of
> a file is sparse, without having to read every byte within that region.
>  Now that this series is giving QED special metadata that marks a zero
> cluster, it is faster to query if that metadata exists denoting a zero
> cluster than it is to read the entire cluster and check for non-zero.
> Likewise, with regular files, the kernel provides lseek(SEEK_HOLE) (or
> the older, lower-level, ioctl(FS_IOC_FIEMAP)); which at least GNU
> coreutils is using for efficient sparse detection in source files.

Yes, there are ways to optimize this for specific storage backends.  But
we need a code path that supports all storage systems first.  For
example, raw files over NFS or an image file over HTTP (curl).

In the case of qcow2 or QED backing files we already don't read zeroes
today.  Instead we memset the read buffer to zero and the waste CPU
cycles doing buffer_is_zero() detection.  At least this means that file
I/O (and network I/O, if using NFS) is already optimal if your backing
file is qcow2 or QED - it's just the CPU cycles that we can optimize
away.

Stefan