[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 0/5] vfio/pci: Fix up breakage against split irqchip and I
From: |
Auger Eric |
Subject: |
Re: [PATCH v2 0/5] vfio/pci: Fix up breakage against split irqchip and INTx |
Date: |
Mon, 2 Mar 2020 16:09:59 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 |
Hi Peter,
On 2/28/20 5:14 PM, Peter Xu wrote:
> v2:
> - pick tags
> - don't register resamplefd with KVM kernel when the userspace
> resamplefd path is enabled (should enable fast path on new kernels)
> - fix resamplefd mem leak
> - fix commit message of patch 4 [Eric]
> - let kvm_resample_fd_notify() return a boolean, skip ioapic check if
> returned true
> - more comments here and there in the code to state the fact that
> userspace ioapic irr & remote-irr are bypassed [Paolo]
>
> VFIO INTx is not working with split irqchip. On new kernels KVM_IRQFD
> will directly fail with resamplefd attached so QEMU will automatically
> fallback to the INTx slow path. However on old kernels it's still
> broken.
>
> Only until recently I noticed that this could also break PXE boot for
> assigned NICs [1]. My wild guess is that the PXE ROM will be mostly
> using INTx as well, which means we can't bypass that even if we
> enables MSI for the guest kernel.
>
> This series tries to first fix this issue function-wise, then speed up
> for the INTx again with resamplefd (mostly following the ideas
> proposed by Paolo one year ago [2]). My TCP_RR test shows that:
>
> - Before this series: this is broken, no number to show
>
> - After patch 1 (enable slow path): get 63% perf comparing to full
> kernel irqchip
>
> - After whole series (enable fast path partly, irq injection will be
> the same as fast path, however userspace needs to intercept for
> EOI broadcast to resamplefd, though should still be faster than
> the MMIO trick for intx eoi): get 93% perf comparing to full
> kernel irqchip, which is a 46% performance boost
>
> I think we can consider to apply patch 1 even sooner than the rest of
> the series to unbreak intx+split first.
>
> The whole test matrix for reference:
>
>
> |----------+---------+-------------------+--------------+--------------------|
> | IRQ type | irqchip | TCP_STREAM (Gbps) | TCP_RR (pps) | note
> |
>
> |----------+---------+-------------------+--------------+--------------------|
> | msi | on | 9.39 | 17567 |
> |
> | nomsi | on | 9.29 | 14056 |
> |
> | msi | split | 9.36 | 17330 |
> |
> | nomsi | split | / | / | currently broken
> |
> | nomsi | split | 8.98 | 8977 | after patch 1
> |
> | nomsi | split | 9.21 | 13142 | after whole
> series |
>
> |----------+---------+-------------------+--------------+--------------------|
>
> Any review comment is welcomed. Thanks,
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1786404
> [2] https://patchwork.kernel.org/patch/10738541/#22609933
For the whole series:
Tested-by: Eric Auger <address@hidden>
using a 5.2-rc1 kernel with reverted "654f1f13ea56 kvm: Check
irqchip mode before assign irqfd" and guest booting with nomsi.
Thanks
Eric
>
> Peter Xu (5):
> vfio/pci: Disable INTx fast path if using split irqchip
> vfio/pci: Use kvm_irqchip_add_irqfd_notifier_gsi() for irqfds
> KVM: Pass EventNotifier into kvm_irqchip_assign_irqfd
> KVM: Kick resamplefd for split kernel irqchip
> Revert "vfio/pci: Disable INTx fast path if using split irqchip"
>
> accel/kvm/kvm-all.c | 101 +++++++++++++++++++++++++++++++++++++----
> accel/kvm/trace-events | 1 +
> hw/intc/ioapic.c | 23 +++++++++-
> hw/vfio/pci.c | 37 ++++++---------
> include/sysemu/kvm.h | 7 +++
> 5 files changed, 137 insertions(+), 32 deletions(-)
>
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: [PATCH v2 0/5] vfio/pci: Fix up breakage against split irqchip and INTx,
Auger Eric <=