Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores

[Michael S. Tsirkin]

On Tue, Nov 13, 2012 at 05:21:50PM +0100, Peter Lieven wrote:
> On 13.11.2012 17:22, Michael S. Tsirkin wrote:
> >On Tue, Nov 13, 2012 at 12:49:03PM +0100, Peter Lieven wrote:
> >>On 09.11.2012 19:03, Peter Lieven wrote:
> >>>Remark:
> >>>If i disable interrupts on CPU1-3 for virtio the performance is ok again.
> >>>
> >>>Now we need someone with deeper knowledge of the in-kernel irqchip and the
> >>>virtio/vhost driver development to say if this is a regression in qemu-kvm
> >>>or a problem with the old virtio drivers if they receive the interrupt on
> >>>different CPUs.
> >>anyone?
> >Looks like the problem is not in the guest: I tried ubuntu guest
> >on a rhel host, I got 8GB/s with vhost and 4GB/s without
> >on a host to guest banchmark.
> which ubuntu version in the guest?
> 
> Peter

ubuntu-10.04.4-server-amd64.iso

> >
> >
> >
> >>>Peter Lieven wrote:
> >>>>it seems that with in-kernel irqchip the interrupts are distributed across
> >>>>all vpcus. without in-kernel irqchip all interrupts are on cpu0. maybe
> >>>>this is related.
> >>>>
> >>>>without inkernel irqchip
> >>>>            CPU0       CPU1       CPU2       CPU3
> >>>>   0:         16          0          0          0   IO-APIC-edge      
> >>>> timer
> >>>>   1:         23          0          0          0   IO-APIC-edge      
> >>>> i8042
> >>>>   4:          1          0          0          0   IO-APIC-edge
> >>>>   6:          4          0          0          0   IO-APIC-edge
> >>>>floppy
> >>>>   7:          0          0          0          0   IO-APIC-edge
> >>>>parport0
> >>>>   8:          0          0          0          0   IO-APIC-edge      rtc0
> >>>>   9:          0          0          0          0   IO-APIC-fasteoi   acpi
> >>>>  11:         76          0          0          0   IO-APIC-fasteoi
> >>>>uhci_hcd:usb1
> >>>>  12:        102          0          0          0   IO-APIC-edge      
> >>>> i8042
> >>>>  14:          0          0          0          0   IO-APIC-edge
> >>>>ata_piix
> >>>>  15:      16881          0          0          0   IO-APIC-edge
> >>>>ata_piix
> >>>>  24:          0          0          0          0   PCI-MSI-edge
> >>>>virtio1-config
> >>>>  25:       5225          0          0          0   PCI-MSI-edge
> >>>>virtio1-requests
> >>>>  26:          0          0          0          0   PCI-MSI-edge
> >>>>virtio0-config
> >>>>  27:      72493          0          0          0   PCI-MSI-edge
> >>>>virtio0-input
> >>>>...
> >>>>
> >>>>with inkernel irqchip
> >>>>            CPU0       CPU1       CPU2       CPU3
> >>>>   0:         16          0          0          0   IO-APIC-edge      
> >>>> timer
> >>>>   1:          0          3          3          1   IO-APIC-edge      
> >>>> i8042
> >>>>   4:          0          0          1          0   IO-APIC-edge
> >>>>   6:          1          0          1          2   IO-APIC-edge
> >>>>floppy
> >>>>   7:          0          0          0          0   IO-APIC-edge
> >>>>parport0
> >>>>   8:          0          0          0          0   IO-APIC-edge      rtc0
> >>>>   9:          0          0          0          0   IO-APIC-fasteoi   acpi
> >>>>  11:          7          9          4          1   IO-APIC-fasteoi
> >>>>uhci_hcd:usb1
> >>>>  12:         30         27         29         34   IO-APIC-edge      
> >>>> i8042
> >>>>  14:          0          0          0          0   IO-APIC-edge
> >>>>ata_piix
> >>>>  15:        943        937        950        943   IO-APIC-edge
> >>>>ata_piix
> >>>>  24:          0          0          0          0   PCI-MSI-edge
> >>>>virtio0-config
> >>>>  25:        930        978        980        947   PCI-MSI-edge
> >>>>virtio0-input
> >>>>  26:          0          0          1          0   PCI-MSI-edge
> >>>>virtio0-output
> >>>>  27:          0          0          0          0   PCI-MSI-edge
> >>>>virtio1-config
> >>>>  28:        543        541        542        553   PCI-MSI-edge
> >>>>virtio1-requests
> >>>>...
> >>>>
> >>>>