Re: [Qemu-devel] [Qemu-block] Question regarding qemuimg check

[Ala Hino]

Thanks for the detailed reply, Stefan.

Can we always run qemu-img check with -r leaks option, even if there are no
leaks?

On Tue, Oct 24, 2017 at 3:06 PM, Stefan Hajnoczi <address@hidden> wrote:

> On Mon, Oct 23, 2017 at 03:38:40PM +0300, Ala Hino wrote:
> > I have a question regarding qemuimg check. We use qemuimg check in order
> to
> > get the offset of image. we need the offset to reduce the size of the
> image
> > to optimal.
> >
> > In BZ 1502488 <https://bugzilla.redhat.com/1502488>, we are
> encountering a
> > use case where a leaked cluster error when executing qemuimg check. The
> > root cause of that exception is killing qemu-kvm process during writing
> to
> > a VM. In this case, executing qemuimg check ends with getting the leaked
> > cluster error. Below is the error:
> >
> > 2017-10-16 10:09:32,950+0530 DEBUG (tasks/0) [root] /usr/bin/taskset
> > --cpu-list 0-3 /usr/bin/qemu-img check --output json -f qcow2
> > /rhev/data-center/mnt/blockSD/8257cf14-d88d-4e4e-998c-
> 9f8976dac2a2/images/7455de38-1df1-4acd-b07c-9dc2138aafb3/
> be4a4d85-d7e6-4725-b7f5-90c9d935c336
> > (cwd None) (commands:69)
> > 2017-10-16 10:09:33,576+0530 ERROR (tasks/0)
> > [storage.TaskManager.Task]
> > (Task='59404af6-b400-4e08-9691-9a64cdf00374') Unexpected error
> > (task:872)
> > Traceback (most recent call last):
> >   File "/usr/share/vdsm/storage/task.py", line 879, in _run
> >     return fn(*args, **kargs)
> >   File "/usr/share/vdsm/storage/task.py", line 333, in run
> >     return self.cmd(*self.argslist, **self.argsdict)
> >   File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
> > line 79, in wrapper
> >     return method(self, *args, **kwargs)
> >   File "/usr/share/vdsm/storage/sp.py", line 1892, in finalizeMerge
> >     merge.finalize(subchainInfo)
> >   File "/usr/share/vdsm/storage/merge.py", line 271, in finalize
> >     optimal_size = subchain.base_vol.optimal_size()
> >   File "/usr/share/vdsm/storage/blockVolume.py", line 440, in
> optimal_size
> >     check = qemuimg.check(self.getVolumePath(), qemuimg.FORMAT.QCOW2)
> >   File "/usr/lib/python2.7/site-packages/vdsm/qemuimg.py", line 156, in
> check
> >     out = _run_cmd(cmd)
> >   File "/usr/lib/python2.7/site-packages/vdsm/qemuimg.py", line 416, in
> _run_cmd
> >     raise QImgError(cmd, rc, out, err)
> > QImgError: cmd=['/usr/bin/qemu-img', 'check', '--output', 'json',
> > '-f', 'qcow2', '/rhev/data-center/mnt/blockSD/8257cf14-d88d-4e4e-
> 998c-9f8976dac2a2/images/7455de38-1df1-4acd-b07c-
> 9dc2138aafb3/be4a4d85-d7e6-4725-b7f5-90c9d935c336'],
> > ecode=3, stdout={
> > QImgError: cmd=['/usr/bin/qemu-img', 'check', '--output', 'json',
> > '-f', 'qcow2', '/rhev/data-center/mnt/blockSD/8257cf14-d88d-4e4e-
> 998c-9f8976dac2a2/images/7455de38-1df1-4acd-b07c-
> 9dc2138aafb3/be4a4d85-d7e6-4725-b7f5-90c9d935c336'],
> > ecode=3, stdout={
> >     "image-end-offset": 7188578304,
> >     "total-clusters": 180224,
> >     "check-errors": 0,
> >     "leaks": 200,
> >     "leaks-fixed": 0,
> >     "allocated-clusters": 109461,
> >     "filename":
> > "/rhev/data-center/mnt/blockSD/8257cf14-d88d-4e4e-
> 998c-9f8976dac2a2/images/7455de38-1df1-4acd-b07c-
> 9dc2138aafb3/be4a4d85-d7e6-4725-b7f5-90c9d935c336",
> >     "format": "qcow2",
> >     "fragmented-clusters": 16741
> > }
> > , stderr=Leaked cluster 109202 refcount=1 reference=0
> >
> >
> > Based on the error info, "This means waste of disk space, but no harm to
> > data", is it OK to handle the error and continue in the flow as usual?
>
> It may be best to fix the image file so that the same leak errors do not
> appear again later:
>
>   $ qemu-img check -f qcow2 -r leaks path/to/image.qcow2
>
> > When hitting this behavior, the return code is 3. Are there other use
> > cases, in addition to cluster leaks, where 3 is returned as the error
> code?
> > Meaning, can we rely on that return code to determine that it is a leaked
> > cluster failure?
>
> 3 means leaks only, this is documented on the qemu-img man page:
>
>   In case the image does not have any inconsistencies, check exits with
> 0.  Other exit codes indicate the kind of inconsistency found or if another
> error occurred. The following table summarizes
>   all exit codes of the check subcommand:
>
>   0   Check completed, the image is (now) consistent
>
>   1   Check not completed because of internal errors
>
>   2   Check completed, image is corrupted
>
>   3   Check completed, image has leaked clusters, but is not corrupted
>
>   63  Checks are not supported by the image format
>
> > If we would like to ignore the cluster leaks, is there a way to call
> > qemuimg check (with some parameter maybe ?) that will not raise the
> error?
>
> Yes, see the qemu-img check repair command-line I posted above.
>
> > Finally, are we doing the right thing to get the image offset in order to
> > reduce its size to optimal?
> >
> > (If you wonder why we need to reduce the image size, this is because
> during
> > snapshot merge, we extend the image size to accumulate the data of the
> top
> > and the base images.)
>
> Yes, qemu-img check is the way to do this.
>
> You're probably hoping there is another command like "qemu-img info"
> that displays the end of image offset but unfortunately you can only
> collect this information by performing a check (it scans the entire
> image and can therefore produce the end of image offset).
>
> Stefan
>