Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration

[Vladimir Sementsov-Ogievskiy]

On 12.01.2015 17:42, Paolo Bonzini wrote:
> On 12/01/2015 15:20, Vladimir Sementsov-Ogievskiy wrote:
> > On 09.01.2015 01:36, Paolo Bonzini wrote:
> > > The bitmaps are transmitted many times in their entirety, but only the
> > > last copy actually means something. The others are lost. This means
> > > you should use the non-live interface (register_savevm). This will
> > > simplify the code a lot.
> > But what if the dirty bitmap is really big?
> >
> > For example, for 5 Pb drive the bitmap with granularity 65536 will be of
> > 2 Gb size. So, shouldn't it be migrated iteratively for live migration?
> But your code is not attempting to do that.  It is not attempting to
> track the dirtiness of the dirty bitmap, so to speak.
>
> For such a huge storage, in any case, I suspect the solution is to not
> use QEMU's live operations and, instead, operate at the storage level.
>
> Paolo

On 12.01.2015 22:09, Paolo Bonzini wrote:
> On 12/01/2015 18:31, John Snow wrote:
> > > How do you track which parts of the disk have been acted upon (e.g.
> > > mirrored in the case of the drive-mirror command), so that they have
> > > become 0?
> > Hm... so your question here is, "What happens if the named bitmap you
> > are transferring  gets reset to 0 during migration? Will anybody track
> > this change?"
> >
> > As far as I know, the only "consumer" of _named_ BdrvDirtyBitmaps is
> > block/backup.c, and we do not support it for mirror (yet?)
> Right.
>
> > So can a block-backup job be running WHILE we migrate?
> I think so.
>
> Paolo

There is no dirty bitmap for dirty bitmap in my solution, actually block 
migration dirty bitmap is used for this. It provides tracking of setting 
bitmap bits to 1 (when corresponding blocks are written), and if the 
bitmap changes only is this way (no other block-jobs are working with 
the migrated bitmap) - then the bitmap will be migrated iteratively as 
it is.

Such approach doesn't provide tracking of resetting the bitmap to 0, and 
such changes are not migrated. So if the bitmap is in use by any 
block-job, it may have additional set bits after migration.

If we want to migrate more precisely we need separate dirty bitmap for 
every migrated bitmap. And tracking of this dirty-dirty bitmap should be 
injected into BdrvDirtyBitmap interface.

But do we really need precise migration of the bitmap being in use by 
some block-job (backup for example)? In this case we should "migrate" 
the backup process to, migrate working block-job.. Is it possible and is 
it planned to be implemented?

Best regards,
Vladimir