Re: [Qemu-devel] Help on TLB Flush

[Lluís Vilanova]

Mark Burton writes:

>> On 13 Feb 2015, at 08:24, Peter Maydell <address@hidden> wrote:
>> 
>> On 13 February 2015 at 07:16, Mark Burton <address@hidden> wrote:
>>> If the kernel is doing this - then effectively - for X86, each CPU only
>>> flush’s it’s own TLB (from the perspective of Qemu) - correct?
>>> (in which case, for Qemu itself - for x86) - we dont need to implement
>>> a global flush, and hence we dont need to build the mechanism to sync ?

>> The semantics you need are "flush the QEMU TLB for CPU X" (where
>> X may not be the CPU you're running on). This is what tlb_flush()
>> does: it takes a CPU argument to act on. (Ditto tlb_flush_page, etc.)
>> We then use that to implement the target's required semantics
>> (eg in ARM the tlbiall_is_write() function is handled by iterating
>> through all CPUs and calling tlb_flush on them).

> What Lluis implied seemed to be that the kernel arranged to signal the CPU 
> that would flush. Hence, (for X86), we would only ever flush our own TLB.

That's correct.

[...]
> For our immediate concern, in the interests of getting the thing working and
> making sure we’ve turned over all the stones, on ARM - it MAY help us to check
> that the flush has happened ‘in the next memory barrier’….
>       - I dont know if that will help us or not, and - even if it does, I 
> agree with you, it would be more messy than it need be.
> However, in the interests of making sure that there are no other issues - we 
> may ‘hack’ something before we put in place a more elegant solution…. 
> (right now, we have some mutex issues, shifting the sync to the barrier MAY 
> help us avoid that…. To Be Seen…. and anyway - it would only be a temporary 
> fix).

But you shouldn't assume that everyone either uses x86's semantics (aka, each
CPU gets an IPI), or the ARM semantics you described where the global TLB flush
instruction has asynchronous effects. First, in ARM you still have to ensure
other CPUs did what you asked them to (whenever the arch manual says you must do
so). Second, it seems like ARM does not always behave in the way you described:

  http://lxr.free-electrons.com/source/arch/arm/kernel/smp.c?v=2.6.32#L630

Granted, this is just the same behaviour as x86, but noone guarantees you that
some other operation in any of the multiple architectures supported by QEMU will
never need a synchronous instruction with global effects.

I understand the pressure of getting something running and work from that, but I
think that having a framework for asynchronous cross-CPU messaging would be
rather useful in the future. That can be then complemented with a mechanism to
wait for these asynchronous messages. You can achieve any desired behaviour by
composing these two.


Cheers,
  Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth