Re: [Qemu-devel] [PATCH] Disk image shared and exclusive locks.

[Daniel P. Berrange]

On Mon, Dec 07, 2009 at 07:32:16AM -0600, Anthony Liguori wrote:
> Chris Webb wrote:
> >Hi. There's a connected discussion on the sheepdog list about locking, and 
> >I
> >have a patch there which could complement this one quite well.
> >
> >Sheepdog is a distributed, replicated block store being developed
> >(primarily) for Qemu. Images have a mandatory exclusive locking 
> >requirement,
> >enforced by the cluster manager. Without this, the replication scheme
> >breaks down and you can end up with inconsistent copies of the block
> >image.
> >
> >The initial release of sheepdog took these locks in the block driver
> >bdrv_open() and bdrv_close() hooks. They also added a bdrv_closeall() and
> >ensured it was called in all the usual qemu exit paths to avoid stray 
> >locks.
> >(The rarer case of crashing hosts or crashing qemus will have to be handled
> >externally, and is 'to do'.)
> >
> >The problem was that this prevented live migration, because both ends 
> >wanted
> >to open the image at once, even though only one would be using it at a 
> >time.
> >  
> Yeah, this is a bigger problem I think.  Technically speaking, when 
> using NFS as the backing filesystem, we really should not open the 
> destination end before we close the source end to keep the caches fully 
> coherent.
> 
> I've resisted this because I'm concerned that if we delay the opening of 
> the file on the destination, it could fail.  That's a very late failure 
> and that makes me uncomfortable as just a work around for NFS.

The only other alternative would be for the destination to open the disks,
but not immediately acquire the locks. In the final stage of migration have
the source release its lock & signal to the dest that it can now acquire
the lock. The assumption being that the lock acquisition is far less likely
to fail than the open(), so we focus on making sure we can properly handle
open() failure.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|