qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores


From: Davide Guerri
Subject: Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores
Date: Thu, 14 Mar 2013 10:22:22 +0100

I'd like to reopen this thread because this problem is still here and it's 
really annoying.
Another possible work-around is to pin the virtio nic irq to one virtual CPU 
(with /proc/smp_affinity) but (of course) this is still sub-optimal.

Here are some graphs showing the performance of a heavy loaded machine after 
and before a live migration from KVM-1.0 to KVM-1.2.
Host is a Ubuntu 12.10 Kernel 3.5, guest is a Ubuntu 10.04.4 LTS Kernel 2.6.32 

History
-------
CPU time:  
http://s1299.beta.photobucket.com/user/dguerri/media/CPU-10days_zps4a088ff0.png.html
 
CPU Load:  
http://s1299.beta.photobucket.com/user/dguerri/media/Load-10days_zpsff27f212.png.html
NET pps:   
http://s1299.beta.photobucket.com/user/dguerri/media/pps-10days_zps003dd039.png.html
NET Mbps:  
http://s1299.beta.photobucket.com/user/dguerri/media/Mbps-10days_zpsfc3cba8c.png.html

Current
-------
CPU time:  
http://s1299.beta.photobucket.com/user/dguerri/media/CPU-2days_zpsd362cac6.png.html
CPU Load:  
http://s1299.beta.photobucket.com/user/dguerri/media/load-2days_zpsd4b7b50d.png.html
NET pps:   
http://s1299.beta.photobucket.com/user/dguerri/media/pps-2days_zps8f5458c9.png.html
NET Mbps:  
http://s1299.beta.photobucket.com/user/dguerri/media/Mbps-2days_zps299338b9.png.html
 

Red arrows indicate the kvm version change.
Black arrows indicate when I "pinned" the virtio NIC IRQ to only one CPU (that 
machine has 2 virtual cores).
As can be seen the performance penalty is rather high: that machine was almost 
unusable!

Does version 1.3 fixes this issue?

Could someone with the required knowledge look into this, please? 
Please, this is a very nasty bug because I guess I'm not the only one who is 
unable to upgrade all the machines with a (not-so) old kernel... :)

Thanks!

Davide Guerri.




On 09/dic/2012, at 19:38, Alexandre DERUMIER <address@hidden> wrote:

>>> Have you had any further progress on this regression/problem?
> 
> Hi Peter,
> I didn't re-tested myself,
> but a proxmox user who's have the problem with qemu-kvm 1.2, with windows 
> guest and linux guest,
> don't have the problem anymore with qemu 1.3.
> 
> http://forum.proxmox.com/threads/12157-Win2003R2-in-KVM-VM-is-slow-in-PVE-2-2-when-multiply-CPU-cores-allowed
> 
> I'll try to redone test myself this week
> 
> Regards,
> 
> Alexandre
> 
> ----- Mail original -----
> 
> De: "Peter Lieven" <address@hidden>
> À: "Alexandre DERUMIER" <address@hidden>
> Cc: "Dietmar Maurer" <address@hidden>, "Stefan Hajnoczi" <address@hidden>, 
> "Jan Kiszka" <address@hidden>, address@hidden, "Michael S. Tsirkin" 
> <address@hidden>, "Peter Lieven" <address@hidden>
> Envoyé: Lundi 3 Décembre 2012 12:23:11
> Objet: Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores
> 
> 
> Am 16.11.2012 um 12:00 schrieb Alexandre DERUMIER <address@hidden>:
> 
>>>> While trying to reproduce the bug, we just detected that it depends on the 
>>>> hardware (mainboard) you run on.
>>>> 
>>>> Sigh :-/
>> 
>> Hi,
>> 
>> I can reproduce the bug on all my dell servers,differents generation (R710 
>> (intel),R815 (amd), 2950 (intel).
>> 
>> They all use broadcom bnx2 network card (don't know if it can be related)
>> 
>> host kernel : rhel 63 with 2.6.32 kernel
>> 
>> guest kernel : 2.6.32 (debian squeeze, ubuntu).
>> 
>> No problem with guest kernel 3.2
> 
> Have you had any further progress on this regression/problem?
> 
> Thanks,
> Peter
> 
>> 
>> 
>> 
>> 
>> ----- Mail original -----
>> 
>> De: "Dietmar Maurer" <address@hidden>
>> À: "Peter Lieven" <address@hidden>
>> Cc: "Stefan Hajnoczi" <address@hidden>, "Peter Lieven" <address@hidden>, 
>> "Jan Kiszka" <address@hidden>, address@hidden, "Michael S. Tsirkin" 
>> <address@hidden>
>> Envoyé: Vendredi 16 Novembre 2012 11:44:26
>> Objet: Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores
>> 
>>>> I only tested with RHEL6.3 kernel on host.
>>> 
>>> can you check if there is a difference on interrupt delivery between those
>>> two?
>>> 
>>> cat /proc/interrupts should be sufficient after some traffic has flown. 
>> 
>> While trying to reproduce the bug, we just detected that it depends on the 
>> hardware (mainboard) you run on.
>> 
>> Sigh :-/
> 
> 

Attachment: smime.p7s
Description: S/MIME cryptographic signature


reply via email to

[Prev in Thread] Current Thread [Next in Thread]