Re: bitmap migration bug with -drive while block mirror runs

[John Snow]

On 10/4/19 4:33 AM, Peter Krempa wrote:
> On Thu, Oct 03, 2019 at 19:34:56 -0400, John Snow wrote:
>> On 10/3/19 6:14 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> 03.10.2019 0:35, John Snow wrote:
>>>> On 10/2/19 6:46 AM, Peter Krempa wrote:
>>> ====
> 
> [...]
> 
> (I'm sorry if I ignored something which might require input in the
> trimmed part but I don't have enough mental capacity to follow this
> thread fully)
> 

Yeah, understandable -- it's getting a bit long, but I'm trying to make
sure I understand the nuance everywhere before I start pursuing a
particular solution.

I think you caught the important parts in your replies below.

>>>
>>> If it's a problem for libvirt to keep same node-names, why should we insist?
>>>
>>>
>>
>> Is it really a problem? If libvirt requests migration of bitmaps, those
>> bitmaps need to go somewhere. Even if it constructs a different graph
>> for valid reasons, it should still understand which qcow2 nodes
>> correlate to which other qcow2 nodes and name them accordingly.
> 
> Well, no it is not a problem. Since bitmap migration has a migration
> capability and libvirt by default disables all unknown migration
> capabilities we can deal with it.
> 
> We have measures to transfer state to the destination we can
> basically do the equivalent of the explicit mapping but with extra
> steps.
> 
> We know where we want to place the bitmap and thus we can configure
> those nodes appropriately and generate new names for everything else so
> that nothing gets accidentally copied to wrong place.
> 
> My concern is though about the future. Since this is the first instance
> of such a  migration feature which requires node names it's okay because
> we can cheat by naming the destination "appropriately". The problem
> will start though if there will be something else bound to the backend
> of a disk addressed by node names which will have different semantics.
> 
> In that case we won't be able to cheat again.
> 

OK, I see the concern now. Though we're free to name nodes to achieve
the bitmap semantics we want right now, graph reconfigurations in the
future might not be able to fit within the same constraints simultaneously.

Reasonable concern.

Thank you for the illustrative hypothetical.

> Let's assume the following example:
> 
> qemu adds a new feature of migrating the qcow2 L2 cache. This will
> obviously have different implications on when it can be used than
> bitmaps.
> 
> If we'd like to use either of the features but not both together on a
> node there wouldn't be a possibility to achieve that.
> 
> The thing about bitmaps is that they are not really bound to the image
> itself but rather the data in the image. As long as the user provides a
> image with exactly the same contents the VM can use it and the bitmap
> will be correct for it.
> 
> We use this in non-shared storage migration where we by default flatten
> the backing chain into a new image. In such case a bitmap is still valid
> but the cache in the hypothetical example is not valid to be copied over
> for the same node name.
> 
> At the very least the nuances of the capability should be documented so
> that we don't have to second guess what is going to happen.
> 

OK, understood.

>> I don't see why this is actually a terrible constraint. Even in our
>> mapping proposal we're still using node-names.
>>
>>
>> So here's a summary of where I stand right now:
>>
>> - I want an API in QEMU that doesn't require libvirt.
>>
>> - I want to accommodate libvirt, but don't understand the requirement
>> that node-names must be ephemeral.
> 
> As I've outlined above, the node names must not be ephemeral but on the
> same note it's then necessary to clarify when they must be stable
> accross migration and when they must be different.
> 
> In the above example I'm outlining an image which has the same data but
> it's a different image (it was converted for example). In that case the
> bitmap migration would imply the same node name, but at the same time
> the image is completely different and any other feature may be
> incompatible with it.
> 
> The same is possible e.g. when you have multiple protocols to access the
> same data are they the same thing and thus warrant the same node name?
> or are they different.
> 
> Treating node names as ephemeral has the advantage of not trying to
> assume the equivalence of the images on the migration channel and not
> having to try to figure out whether they are "euqivalent enough" for the
> given feature.
> 
>>
>> - I would like to define a set of default behaviors (when bitmap
>> migration is enabled) that migrates only bitmaps it is confident it can
>> do so correctly and error out when it cannot.
> 
> This requires also defining a set of external constraints when it will
> work. Note that they can differ with other features.
> 
>>
>> - I'd like to amend the bitmap device name resolution to accommodate the
>> drive-mirror case.
>>
>> - Acknowledging that there might be cases where the defaults just simply
>> aren't powerful enough, allow a manual configuration that allows us to
>> select or deselect bitmaps and explicitly set their destination node-name.
> 
> This tangentially brings me to another question. In case when the
> destination image already contains a bitmap with the same name, will the
> migration of bitmaps overwrite it or merge with it?
> 

It will emit an error, currently.

> This is again one thing that should be documented.
> 
> In the outlined case of non-shared storage migration libvirt would
> obviously prefer merge or having it configurable, but as said, we have
> means to work this around by renaming the bitmap temporarily  during
> migration and then merging it explicitly.
> 

Well, we don't know what bitmap is already there or what semantics it
has -- so at the moment we just bail out because we can't tell what's
going on.

If you find a use case for merge or overwrite, we can add that as a
feature, but right now we do the "safe thing".



I would have still liked to find a way to migrate bitmap with a "sane
default", but because I don't know what I don't know, it might be the
case that node-names in the stream that aren't explicitly requested
could be a very limiting factor in the future. Sadly, we are already
using them -- but perhaps we can find a way to back out of that.


So I think the safest thing, given your concern, is:
- Any bitmap attached to a root node can be migrated using the root's
drive name, if it has one. (This includes the drive-mirror case, if we
fix our ability to resolve the root through the filter node.)

- Any bitmap that doesn't have a named device or backend at its root
should not be migrated without further configuration, because doing so
requires node-names in the stream, for which we have a poor
understanding of possible future competing constraints.


I'm a little sad that this means that blockdev configurations can't be
migrated without a much more verbose migration statement.

I'm open to suggestions, but it sounds like we do want the ability to
specify a migration mapping to keep migration as flexible as possible.

I'll sleep on it.

--js