[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Any reason VIRTQUEUE_MAX_SIZE is 1024? Can we increase this limit?

From: Jason Wang
Subject: Re: Any reason VIRTQUEUE_MAX_SIZE is 1024? Can we increase this limit?
Date: Fri, 7 Aug 2020 11:35:08 +0800
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0

On 2020/8/6 下午8:37, Stefan Hajnoczi wrote:
On Wed, Aug 05, 2020 at 08:13:29AM -0400, Michael S. Tsirkin wrote:
On Wed, Aug 05, 2020 at 01:11:07PM +0100, Stefan Hajnoczi wrote:
On Thu, Jul 30, 2020 at 07:46:09AM +0000, Yajun Wu wrote:
I'm doing iperf test on VIRTIO net through vhost-user(HW VDPA).
Find maximal acceptable tx_queue_size/rx_queue_size is 1024.
Basically increase queue size can get better RX rate for my case.

Can we increase the limit(VIRTQUEUE_MAX_SIZE) to 8192 to possibly gain better 
The VIRTIO 1.1 specification says the maximum number of descriptors is
32768 for both split and packed virtqueues.

The vhost kernel code seems to support 32768.

The 1024 limit is an implementation limit in QEMU. Increasing it would
require QEMU code changes. For example, VIRTQUEUE_MAX_SIZE is used as
the size of arrays.

I can't think of a fundamental reason why QEMU needs to limit itself to
1024 descriptors. Raising the limit would require fixing up the code and
ensuring that live migration remains compatible with older versions of

There's actually a reason for a limit: in theory the vq size
also sets a limit on the number of scatter/gather entries.
both QEMU and vhost can't handle a packet split over > 1k chunks.

We could add an extra limit for s/g size like block and scsi do,
this will need spec, guest and host side work.
Interesting, thanks for explaining! This could be made explicit by
changing the QEMU code to:

include/hw/virtio/virtio.h:#define VIRTQUEUE_MAX_SIZE IOV_MAX

Looking more closely at the vhost kernel code I see that UIO_MAXIOV is
used in some places but not in vhost_vring_set_num() (ioctl
VHOST_SET_VRING_NUM). Is there a reason why UIO_MAXIOV isn't enforced
when the application sets the queue size?

Actually three things:

1) queue size
2) #descriptors in a list
3) IOV size

Spec limit the 2) to 1) but 2) may not equal to 3).

So enforcing UIO_MAXIOV can not solve the problem completely.

For vhost-net, it depends on socket to build skb which requires an iov array to work. We need remove this limitation by:

- build skb by vhost-net itself
- do piecewise copying

Then we're not limited with #iov and more and support up to what spec supports.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]