Re: [Qemu-devel] [PATCH] hw/arm/virt: Add always-on property to the virt board timer

[Marc Zyngier]

On 20/01/16 14:01, Andrew Jones wrote:
> On Tue, Jan 19, 2016 at 07:48:14PM +0100, Andrew Jones wrote:
>> On Tue, Jan 19, 2016 at 01:43:07PM +0000, Marc Zyngier wrote:
>>>>> On Tue, Jan 19, 2016 at 01:37:16PM +0100, Andrew Jones wrote:
>>>> OK, CCing him. One thing I see is that without this change we're
>>>> currently setting the clock feature CLOCK_EVT_FEAT_C3STOP, even though
>>>> it's not true. Having that set may disable the oneshot capabilityj
>>>> necessary to switch to nohz mode? I'll just stop there with my
>>>> speculation though, so Marc won't have to correct too much...
>>>
>>> You're spot on. See 82a5619 in the kernel tree. When I did a similar
>>> change in kvmtool, I saw a massive reduction in the number of timer
>>> interrupts injected (specially when the number of vcpu is relatively high).
>>>
>>> This also have interesting benefits when running on a model, where
>>> you're trying to squeeze the last bits of "performance" from the monster...
>>>
>>
>> Hmm, I'm probably testing this wrong, but I don't see any difference in
>> the number of injected timer interrupts. My guest, which I boot with
>> UEFI, has 
>>
>> CONFIG_ARM_ARCH_TIMER=y
>> CONFIG_ARM_ARCH_TIMER_EVTSTREAM=y
>> CONFIG_ARM_TIMER_SP804=y
>> CONFIG_HIGH_RES_TIMERS=y
>> CONFIG_TICK_ONESHOT=y
>> CONFIG_NO_HZ_COMMON=y
>> # CONFIG_HZ_PERIODIC is not set
>> CONFIG_NO_HZ_IDLE=y
>> # CONFIG_NO_HZ_FULL is not set
>> CONFIG_NO_HZ=y
>> CONFIG_HZ_1000=y
>> CONFIG_HZ=1000
>>
>> I've boot a guest using DT with and without this patch
>>
>> ---WITHOUT---
>>
>> # ls /proc/device-tree/timer
>> compatible  interrupts  name
>> # cat /proc/interrupts                  
>>            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5 CPU6  
>>      CPU7
>>   3:       6958       5766       5166       5187       5576       5129 4695  
>>      4398       GIC  27 Edge      arch_timer
>> # sleep 120 && cat /proc/interrupts                  
>>            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5 CPU6  
>>      CPU7
>>   3:       7557       5986       5487       5265       6232       5868 5464  
>>      4438       GIC  27 Edge      arch_timer
>>
>> ---WITH---
>>
>> # ls /proc/device-tree/timer
>> always-on  compatible  interrupts  name
>> # cat /proc/interrupts 
>>            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5 CPU6  
>>      CPU7
>>   3:       7005       6080       4996       5391       5165       5257 4930  
>>      4844       GIC  27 Edge      arch_timer
>> # sleep 120 && cat /proc/interrupts 
>>            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5 CPU6  
>>      CPU7
>>   3:       7523       6505       5264       6717       5273       5391 5526  
>>      4901       GIC  27 Edge      arch_timer
>>
>>
>>
>> And kvm trace data has
>>
>> ---WITHOUT---
>> $ grep kvm_timer_update_irq trace.out | wc -l
>> 94336
>> ---WITH---
>> $ grep kvm_timer_update_irq trace.out | wc -l
>> 95838
>>
>>
> 
> Must be how I'm looking, because I just tried kvmtool with/without
> Marc's patch that adds always-on, but don't see any reduction of
> interrupts there either. I used a defconfig guest kernel. Also,
> not that I think it should matter, but my host kernel is 4.4-rc4
> based.
> 
> I'd like to be able to see a difference with/without this always-on
> patch, not because I don't think we should take it anyway, but because
> I need a test case for the ACPI counterpart.

I just run a couple of quick tests, measuring interrupt rate (vmstat 1)
on the host, with one VM (2 vcpus) idling, and I'm seeing the following
thing:

Without "always-on": ~380 interrupts per second
With "always-on": ~40 interrupts per second

This is with kvmtool, 32bit host (but none of that is arch specific anyway).

        M.
-- 
Jazz is not dead. It just smells funny...