[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [RFC] dataplane: IOThreads and writing dataplane-capabl
From: |
Stefan Hajnoczi |
Subject: |
Re: [Qemu-devel] [RFC] dataplane: IOThreads and writing dataplane-capable code |
Date: |
Fri, 9 May 2014 10:20:32 +0200 |
User-agent: |
Mutt/1.5.21 (2010-09-15) |
On Thu, May 08, 2014 at 07:58:11PM +0100, Dr. David Alan Gilbert wrote:
> * Stefan Hajnoczi (address@hidden) wrote:
> > On Thu, May 8, 2014 at 3:44 PM, Dr. David Alan Gilbert
> > <address@hidden> wrote:
> > > * Stefan Hajnoczi (address@hidden) wrote:
> > >
> > > <snip>
> > >
> > >> How to synchronize with an IOThread
> > >> -----------------------------------
> > >> AioContext is not thread-safe so some rules must be followed when using
> > >> file
> > >> descriptors, event notifiers, timers, or BHs across threads:
> > >>
> > >> 1. AioContext functions can be called safely from file descriptor, event
> > >> notifier, timer, or BH callbacks invoked by the AioContext. No locking
> > >> is
> > >> necessary.
> > >>
> > >> 2. Other threads wishing to access the AioContext must use
> > >> aio_context_acquire()/aio_context_release() for mutual exclusion. Once
> > >> the
> > >> context is acquired no other thread can access it or run event loop
> > >> iterations
> > >> in this AioContext.
> > >>
> > >> aio_context_acquire()/aio_context_release() calls may be nested. This
> > >> means you can call them if you're not sure whether #1 applies.
> > >>
> > >> Side note: the best way to schedule a function call across threads is to
> > >> create
> > >> a BH in the target AioContext beforehand and then call
> > >> qemu_bh_schedule(). No
> > >> acquire/release or locking is needed for the qemu_bh_schedule() call.
> > >> But be
> > >> sure to acquire the AioContext for aio_bh_new() if necessary.
> > >
> > > How do these IOThreads pause during migration?
> > > Are they paused by the 'qemu_mutex_lock_iothread' that the migration
> > > thread calls?
> >
> > Currently the only IOThread user is virtio-blk data-plane. It has a
> > VM state change listener registered that will stop using the IOThread
> > during migration.
> >
> > In the future we'll have to do more than that:
> > It is possible to suspend all IOThreads simply by looping over
> > IOThread objects and calling aio_context_acquire() on their
> > AioContext. You can release the AioContexts when you are done. This
> > would be suitable for a "stop the world" operation for migration
> > hand-over.
>
> That worries me for two reasons:
> 1) I'm assuming there is some subtlety so that it doesn't deadlock when
> another thread is trying to get a couple of contexts.
Only the main loop acquires contexts, that's why there is no lock
ordering problem.
> 2) The migration code that has to pause everything is reasonably time
> critical (OK not super critical - but it worries if it gains more than
> a few
> ms). Doing something to each thread in series where that thread might
> have to finish up a transaction sounds like it could add together to be
> quite
> large.
It's no different from today where we need to bdrv_drain_all();
bdrv_flush_all(). That's a synchronous operation that can take a while.
> > For smaller one-off operations like block-migration.c it may also make
> > sense to acquire/release the AioContext. But that's not necessary
> > today since dataplane is disabled during migration.
>
> I guess it's probably right to hide this behind some interface on the Aio
> stuff
> that migration can call and it can worry about speed, and locking order etc.
>
> I also would we end up wanting some IOThreads to continue - e.g. could we be
> using
> them for transport of the migration stream or are they strictly for the guests
> use?
IOThreads are just threads running AioContext event loops. They are
generic and could be used for stuff I/O intensive stuff like migration
or the VNC server.
Stefan