Re: [PATCH v2] Document qemu-img options data_file and data_file_raw

[Max Reitz]

On 30.04.21 15:34, Connor Kuehl wrote:
> The contents of this patch were initially developed and posted by Han
> Han[1], however, it appears the original patch was not applied. Since
> then, the relevant documentation has been moved and adapted to a new
> format.
>
> I've taken most of the original wording and tweaked it according to
> some of the feedback from the original patch submission. I've also
> adapted it to restructured text, which is the format the documentation
> currently uses.
>
> [1] https://lists.nongnu.org/archive/html/qemu-block/2019-10/msg01253.html
>
> Fixes: https://bugzilla.redhat.com/1763105
> Signed-off-by: Han Han <hhan@redhat.com>
> Signed-off-by: Connor Kuehl <ckuehl@redhat.com>
> ---
> Changes since v1:
>    * Clarify different behaviors with these options when using qemu-img
>      create vs amend (Max)
>    * Touch on the negative case of how the file becomes inconsistent
>      (John)
>
>   docs/tools/qemu-img.rst | 20 ++++++++++++++++++++
>   1 file changed, 20 insertions(+)
>
> diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
> index c9efcfaefc..87b4a65535 100644
> --- a/docs/tools/qemu-img.rst
> +++ b/docs/tools/qemu-img.rst
> @@ -866,6 +866,26 @@ Supported image file formats:
>       issue ``lsattr filename`` to check if the NOCOW flag is set or not
>       (Capital 'C' is NOCOW flag).
>   
> +  ``data_file``
> +    Filename where all guest data will be stored. If this option is used,
> +    the qcow2 file will only contain the image's metadata.
> +
> +    Note: Data loss will occur if the given filename already exists when
> +    using this option with ``qemu-img create`` since ``qemu-img`` will create
> +    the data file anew, overwriting the file's original contents. To simply
> +    update the reference to point to the given pre-existing file, use
> +    ``qemu-img amend``.
> +
> +  ``data_file_raw``
> +    If this option is set to ``on``, QEMU will always keep the external
> +    data file consistent as a standalone read-only raw image. It does
> +    this by forwarding updates through to the raw image in addition to
> +    updating the image metadata. If set to ``off``, QEMU will only
> +    update the image metadata without forwarding the changes through
> +    to the raw image. The default value is ``off``.

Hm, what updates and what changes?  I mean, the first part makes sense 
(the “It does this by...”), but the second part doesn’t.  qemu will 
still forward most writes to the data file.  (Not all, but most.)

(Also, nit pick: With data_file_raw=off, the data file is not a raw 
image.  (You still call it that in the penultimate sentence.))

When you write data to a qcow2 file with data_file, the data also goes 
to the data_file, most of the time.  The exception is when it can be 
handled with a metadata update, i.e. when it's a zero write or discard.

In addition, such updates (i.e. zero writes, I presume) not happening to 
the data file are usually a minor problem.  The real problem is that 
without data_file_raw, data clusters can be allocated anywhere in the 
data file, whereas with data_file_raw, they are allocated at their 
respective guest offset (i.e. the host offset always equals the guest 
offset).

I personally would have been fine with the first sentence, but if we 
want more of an explanation...  Perhaps:

<<EOF

If this option is set to ``on``, QEMU will always keep the external data 
file consistent as a standalone read-only raw image.

It does this by effectively forwarding all write accesses that happen to 
the qcow2 file to the raw data file, including their offsets. 
Therefore, data that is visible on the qcow2 node (i.e., to the guest) 
at some offset is visible at the same offset in the raw data file.

If this option is ``off``, QEMU will use the data file just to store 
data in an effectively arbitrary manner.  The file’s content will not 
make sense without the accompanying qcow2 metadata.  Where data is 
written will have no relation to its offset as seen by the guest, and 
some writes (specifically zero writes) may not be forwarded to the data 
file at all, but will only be handled by modifying qcow2 metadata.

In short: With data_file_raw, the data file reads as a valid raw VM 
image file.  Without it, its content can only be interpreted by reading 
the accompanying qcow2 metadata.

Note that this option only makes the data file valid as a read-only raw 
image.  You should not write to it, as this may effectively corrupt the 
qcow2 metadata (for example, dirty bitmaps may become out of sync).

EOF

This got longer than I wanted it to be.  Hm.  Anyway, what do you think?

Max

> +
> +    This option can only be enabled if ``data_file`` is set.
> +
>   ``Other``
>   
>     QEMU also supports various other image file formats for