[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [RFC PATCH v1 7/9] cpus: move icount preparation out of

From: Alex Bennée
Subject: Re: [Qemu-devel] [RFC PATCH v1 7/9] cpus: move icount preparation out of tcg_exec_cpu
Date: Wed, 05 Apr 2017 12:18:29 +0100
User-agent: mu4e 0.9.19; emacs 25.2.13

Pavel Dovgalyuk <address@hidden> writes:

>> From: Alex Bennée [mailto:address@hidden
>> Paolo Bonzini <address@hidden> writes:
>> > On 04/04/2017 14:31, Alex Bennée wrote:
>> >>
>> >> Paolo Bonzini <address@hidden> writes:
>> >>
>> >>> On 04/04/2017 12:46, Alex Bennée wrote:
>> >>>>> In theory the main-loop should be sequenced before or after vCPU events
>> >>>>> because of the BQL. I'm not sure why this is not currently the case.
>> >>>>
>> >>>> It seems cpu_handle_exception doesn't take the BQL until
>> >>>> replay_exception() has done its thing. This is fixable but the function
>> >>>> is a mess so I'm trying to neaten that up first.
>> >>>
>> >>> Long term neither cpu_handle_exception nor cpu_handle_interrupt need the
>> >>> BQL at all.
>> >>
>> >> Well for record/replay they might. Otherwise we end up moving the record
>> >> stream on even though a checkpoint might be being written by the
>> >> main-loop.
>> >>
>> >> As far as the cc->do_interrupt() stuff is concerned it will be guest
>> >> dependant because you could end up in device emulation code down this
>> >> path which must be protected by the BQL - the arm_gic code being a good
>> >> example.
>> >
>> > I think recording an event could be split in two parts:
>> >
>> > - recording the (icount, event) tuple and getting back a unique event id
>> >
>> > - waiting for all events with lower event id to be complete before
>> > starting to process this one
>> >
>> > This doesn't require the BQL, you can use a condition variable on
>> > replay_lock (but you do need to unlock/lock the BQL around it if
>> > currently taken).
>> Would you then leave the recording to the stream to the main-loop
>> thread? I guess it would marshal all events that occurred before the
>> checkpoint first and then finish draining the queue after recording its
>> checkpoint?
>> Wrapping the exception stuff in the BQL does improve the repeat-ability
>> but of course it breaks if I take away the graceful handling of time
>> differences because there is a race between recording the exception
>> event (with current_step+insns so far) and getting back to the main loop
>> where insns is finally credited to timers_state.qemu_icount.
>> I guess we could improve the situation by updating
>> timers_state.qemu_icount (under BQL) as we record events. I don't know
>> how clunky that would get.
> Does io instructions make some lock to prevent races in virtual
> hardware?

Yes the BQL is taken for MMIO operations unless the hardware explicitly
does its own locking. Other operations that trigger hardware emulation
(for example changing ARM_CP_IO registers) should also take the BQL.

> vCPU thread updates icount in the beginning of the TB execution.
> It means that checkpoints in the replay log will appear only at the boundaries
> of TBs.

Not quite. The vCPU thread takes into account "in flight" instructions
when calculating the icount. In the next series I'm about to post I've
ensured this reflects an update to the main-loop icount when we read the

> However, the same log may be generated by different scenarios.
> Consider the following cases:
> 1. Sequence: vCPU-block-begin vCPU-update-icount iothread-io vCPU-io 
> vCPU-block-end
> 2. Sequence: vCPU-block-begin vCPU-update-icount vCPU-io iothread-io 
> vCPU-block-end
> These sequences will generate the same order of replay events, but different
> states of virtual hardware.
> Therefore we need some lock for the time while vCPU executes translation block
> (or the whole sequence of blocks as in old times).

Well this can't be the BQL anymore. Using it for serialisation of
multiple threads conflicts with the general aim of reducing BQL
contention across the code-base.

Perhaps we should just push replay_lock up the call-stack to
prepare/process_icount_data take and release the lock and we do the same
in the checkpoint code?

>> > The complicated part is ensuring that there are no deadlocks where the
>> > I/O thread needs the VCPU thread to proceed, but the VCPU thread is
>> > waiting on the I/O thread's event processing.
>> This sort of update sounds more like 2.10 material though.
> Pavel Dovgalyuk

Alex Bennée

reply via email to

[Prev in Thread] Current Thread [Next in Thread]