Re: [Qemu-devel] [PATCH 1/1] util/thread-pool: add parameter to limit maximum threads in thread pool

[felix yao]

Hi stefan:
    Thank you very much for your reply. Yes, I want to reduce impact of
resources among virtual machines including I/O resources, vCPU and etc. I
can use CPU pin feature to pin vCPU to dedicated host's physical CPU and
use I/O throttling feature to limit IOPS and bandwidth of VM's disks. But,
When using aio=threads, I/O worker threads are too many(if we have 16 VMs
in one host, there are 1024 threads at worst) and  can't pinned to
dedicated CPU(dynamic creating). The I/O worker threads seem out of
control. I/O worker threads of one VM's disk may impact the running of
other VM. I am not sure how much of the impact.
    You have mentioned aio=native in the mail. Yes, When Using -drive
aio=native, there is no such problem and performance is better. I am
thinking to use it. But, I found a issue report about using Linux AIO on
QEMU, issue link:
https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/libvirt-aio-mode.html
. http://paste.openstack.org/show/480498/. Do you know about this issue?
Could Linux AIO be safely used in QEMU?
    I have  noticed that there is IOthread feature in QEMU(dedicated thread
to handle disk I/O instead of main thread) recently. I can use 'iothreadpin
iothread=xx cpuset=xxx ' in libvirt XML to pin iothreads of one VM to
dedicated host's physical CPU. Worker threads of the VM are also pinned to
that host's physical CPU. This can also solve my problem.




Stefan Hajnoczi <address@hidden> 于2018年7月27日周五 下午8:41写道：

> On Fri, Jul 13, 2018 at 03:48:27PM +0800, FelixYao wrote:
> > Hi all:
> >
> > Max threads in thread pool is fixed at 64 before which is not
> > propriate in some situations. For public cloud environment,
> > there are lots of VMs in one host machine. We should limit the worker
> thread
> > numbers for each machine alone based on its ability to prevent impacting
> > each other. Although we can use iotune to limit bandwidth or IOPS of
> block
> > device. That is not enough. Lots of worker threads still impact other
> VMs.
> > I think the ability to limit maximum threads in thread pool is necessary.
> >
> > What's your opinion about this patch? please review it.
>
> I haven't seen any discussion so here are some thoughts:
>
> It sounds like you are trying to control I/O resources so that guests do
> not affect each other.  The threads are an implementation detail and not
> the root of the problem.
>
> For example, if you use -drive aio=native then reads and writes will not
> use the thread pool and you will setting the thread pool size won't
> solve the problem.
>
> Have you tried QEMU's I/O throttling feature?  You can set bandwidth
> (bps) and IOPS limits, including bursts if you wish to allow temporary
> spikes.  The basic parameters are -drive bps=BYTES_PER_SECOND,iops=IOPS.
> See the QEMU or libvirt documentation for details.
>