[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] virtio: fix IO request length in virtio SCSI/block #PSBM-788
From: |
Stefan Hajnoczi |
Subject: |
Re: [PATCH] virtio: fix IO request length in virtio SCSI/block #PSBM-78839 |
Date: |
Tue, 12 Nov 2019 10:03:10 +0000 |
User-agent: |
Mutt/1.12.1 (2019-06-15) |
On Wed, Oct 23, 2019 at 05:28:17PM -0400, Michael S. Tsirkin wrote:
> On Fri, Oct 18, 2019 at 02:55:47PM +0300, Denis Plotnikov wrote:
> > From: "Denis V. Lunev" <address@hidden>
> >
> > Linux guests submit IO requests no longer than PAGE_SIZE * max_seg
> > field reported by SCSI controler. Thus typical sequential read with
> > 1 MB size results in the following pattern of the IO from the guest:
> > 8,16 1 15754 2.766095122 2071 D R 2095104 + 1008 [dd]
> > 8,16 1 15755 2.766108785 2071 D R 2096112 + 1008 [dd]
> > 8,16 1 15756 2.766113486 2071 D R 2097120 + 32 [dd]
> > 8,16 1 15757 2.767668961 0 C R 2095104 + 1008 [0]
> > 8,16 1 15758 2.768534315 0 C R 2096112 + 1008 [0]
> > 8,16 1 15759 2.768539782 0 C R 2097120 + 32 [0]
> > The IO was generated by
> > dd if=/dev/sda of=/dev/null bs=1024 iflag=direct
> >
> > This effectively means that on rotational disks we will observe 3 IOPS
> > for each 2 MBs processed. This definitely negatively affects both
> > guest and host IO performance.
> >
> > The cure is relatively simple - we should report lengthy scatter-gather
> > ability of the SCSI controller. Fortunately the situation here is very
> > good. VirtIO transport layer can accomodate 1024 items in one request
> > while we are using only 128. This situation is present since almost
> > very beginning. 2 items are dedicated for request metadata thus we
> > should publish VIRTQUEUE_MAX_SIZE - 2 as max_seg.
> >
> > The following pattern is observed after the patch:
> > 8,16 1 9921 2.662721340 2063 D R 2095104 + 1024 [dd]
> > 8,16 1 9922 2.662737585 2063 D R 2096128 + 1024 [dd]
> > 8,16 1 9923 2.665188167 0 C R 2095104 + 1024 [0]
> > 8,16 1 9924 2.665198777 0 C R 2096128 + 1024 [0]
> > which is much better.
> >
> > The dark side of this patch is that we are tweaking guest visible
> > parameter, though this should be relatively safe as above transport
> > layer support is present in QEMU/host Linux for a very long time.
> > The patch adds configurable property for VirtIO SCSI with a new default
> > and hardcode option for VirtBlock which does not provide good
> > configurable framework.
> >
> > Unfortunately the commit can not be applied as is. For the real cure we
> > need guest to be fixed to accomodate that queue length, which is done
> > only in the latest 4.14 kernel. Thus we are going to expose the property
> > and tweak it on machine type level.
> >
> > The problem with the old kernels is that they have
> > max_segments <= virtqueue_size restriction which cause the guest
> > crashing in the case of violation.
>
> This isn't just in the guests: virtio spec also seems to imply this,
> or at least be vague on this point.
>
> So I think it'll need a feature bit.
> Doing that in a safe way will also allow being compatible with old guests.
The spec is quite explicit about this:
2.6.5 The Virtqueue Descriptor Table
The number of descriptors in the table is defined by the queue size for this
virtqueue: this is the maximum possible descriptor chain length.
and:
2.6.5.3.1 Driver Requirements: Indirect Descriptors
A driver MUST NOT create a descriptor chain longer than the Queue Size of the
device.
If some drivers or devices allow longer descriptor chains today that's
an implementation quirk but a new feature bit is definitely required to
officially allow longer descriptor chains.
Stefan
signature.asc
Description: PGP signature