[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] 答复: migrate_set_speed has no effect if the guest is u
From: |
Dr. David Alan Gilbert |
Subject: |
Re: [Qemu-devel] 答复: migrate_set_speed has no effect if the guest is using hugepages. |
Date: |
Fri, 12 Jul 2019 13:34:00 +0100 |
User-agent: |
Mutt/1.12.0 (2019-05-25) |
* Lin Ma (address@hidden) wrote:
>
>
> > -----邮件原件-----
> > 发件人: Dr. David Alan Gilbert <address@hidden>
> > 发送时间: 2019年7月11日 18:24
> > 收件人: Lin Ma <address@hidden>
> > 抄送: address@hidden
> > 主题: Re: [Qemu-devel] migrate_set_speed has no effect if the guest is using
> > hugepages.
> >
> > * Lin Ma (address@hidden) wrote:
> > > Hi all,
> >
> > Hi Lin,
>
> Hi Dave,
> >
> > > When I live migrate a qemu/kvm guest, If the guest is using huge
> > > pages, I found that the migrate_set_speed command had no effect during
> > stage 2.
> >
> > Can you explain what you mean by 'stage 2'?
> We know that the live migration contains 3 stages:
> Stage 1: Mark all of RAM dirty.
> Stage 2: Keep sending dirty RAM pages since last iteration
> Stage 3: Stop guest, transfer remaining dirty RAM, device state
> (Please refer to
> https://developers.redhat.com/blog/2015/03/24/live-migrating-qemu-kvm-virtual-machines/#live-migration
> for further details)
OK, yeh the numbering is pretty arbitrary so it's not something I
normally think about like that.
>
> > > It was caused by commit 4c011c3 postcopy: Send whole huge pages
> > >
> > > I'm wondering that is it by design or is it a bug waiting for fix?
> >
> > This is the first report I've seen for it. How did you conclude that
> > 4c011c3 caused it? While I can see it might have some effect on the
> > bandwidth management, I'm surprised it has this much effect.
>
> While digging into the bandwidth issue, Git bisect shows that this commit was
> the first bad commit.
OK.
> > What size huge pages are you using - 2MB or 1GB?
>
> When I hit this issue I was using 1GB huge page size.
> I tested this issue with 2MB page size today On Gigabit LAN, Although the
> bandwidth control looks
> a little better than using 1GB, But not too much. Please refer to the below
> test result.
OK, I can certainly see why this might happen with 1GB huge pages; I
need to have a think about a fix.
> > I can imagine we might have a problem that since we only do the sleep
> > between
> > the hugepages, if we were using 1GB hugepages then we'd see <big chunk of
> > data>[sleep]<big chunk of data>[sleep] which isn't as smooth as it used to
> > be.
> >
> > Can you give me some more details of your test?
>
> Live migration bandwidth management testing with 2MB hugepage size:
> sles12sp4_i440fx is a qemu/kvm guest with 6GB memory size.
> Note: the throughput value is approximating value.
>
> Terminal 1:
> virsh migrate-setspeed sles12sp4_i440fx $bandwidth && virsh migrate --live
> sles12sp4_i440fx qemu+tcp://5810f/system
>
> Terminal 2:
> virsh qemu-monitor-command sles12sp4_i440fx --hmp "info migrate"
>
> bandwidth=5
> throughput: 160 mbps
>
> bandwidth=10
> throughput: 167 mbps
>
> bandwidth=15
> throughput: 168 mbps
>
> bandwidth=20
> throughput: 168 mbps
>
> bandwidth=21
> throughput: 336 mbps
>
> bandwidth=22
> throughput: 336 mbps
>
> bandwidth=25
> throughput: 335.87 mbps
>
> bandwidth=30
> throughput: 335 mbps
>
> bandwidth=35
> throughput: 335 mbps
>
> bandwidth=40
> throughput: 335 mbps
>
> bandwidth=45
> throughput: 504.00 mbps
>
> bandwidth=50
> throughput: 500.00 mbps
>
> bandwidth=55
> throughput: 500.00 mbps
>
> bandwidth=60
> throughput: 500.00 mbps
>
> bandwidth=65
> throughput: 650.00 mbps
>
> bandwidth=70
> throughput: 660.00 mbps
OK, so migrate-setspeed takes a bandwidth in MBytes/sec and I guess
you're throughput is in MBit/sec - so at the higher end it's about
right, and at the lower end it's way off.
Let me think about a fix for this.
What are you using to measure throughput?
Dave
>
> Thanks,
> Lin
>
>
> > Dave
> >
> > >
> > > Thanks,
> > > Lin
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
>
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK