Re: [Qemu-devel] QEMU MicroCheckpointing Pause & Resume Latency

[Dr. David Alan Gilbert]

(cc'ing in Paolo since he knows our barrier code)

* FENG, Jiasheng (address@hidden) wrote:
> Dear David,
> 
> Really appreciate your feedback.
> 
> I have proceeded the experiments in both conditions, and no matter the
> vCPUs are in idle or busy situation, there is no difference that smp_wmb()
> will consume a lot of time to proceed its work.
> 
> In your opinion, may I know that what is the alternative way to minimize
> the time consumption of smp_wmb() or any other system setting could speed
> up smp_wmb()?
> 
> Thanks in advance for your assistance and hope to receive your feedback soon

Just checking, is this on a normal x86 PC?
Your numbers of 3-5ms just seem quite high to me but I've not tried timing that
code.

Dave

> 
> Thanks and best regards,
> Niko Jiasheng Feng
> 
> 
> 
> On Thu, Mar 9, 2017 at 11:19 PM, Dr. David Alan Gilbert <address@hidden
> > wrote:
> 
> > * FENG, Jiasheng (address@hidden) wrote:
> > > Dear QEMU Development Team,
> > >
> > >
> > > It is my honor to contact with you.
> > >
> > >
> > >
> > > I am a postgraduate student from University of Hong Kong. Currently I am
> > > working on a project related to QEMU MicroCheckpointing and I have
> > > encountered a performance issue during checkpoint pause & resume.
> >
> > The microcheckpointing code hasn't been maintained for a long time;
> > most of the current checkpointing work is based on the COLO work which is
> > still under development.
> >
> > > Please kindly refer to migration/checkpoint.c file, in function
> > > capture_checkpoint, I proceeded a test to see the time consumption
> > between
> > > vm_stop_force_state and vm_start. I found out that even if the system is
> > > idle, there are still 12-20ms latency recorded ( mem=2G, vCPU=4 ).
> > > Moreover, latency will be increased while more cpus equipped by my
> > virtual
> > > machine. I have done some research on that and I realized that it is
> > > related to the Memory Barrier in KVM kernel. Each cpu will proceed a
> > > smp_wmb() request during pause & resume and it takes about  3-5ms to
> > finish
> > > the request ( mem=2G, vCPU=4 ).
> > >
> > >
> > >
> > > Therefore, I would like to ask 3 questions regarding on the above issue:
> > >
> > >
> > > 1. What is your consideration with calling smp_wmb() in checkpoint
> > period;
> > >
> > > 2. Is it any other solution to minimize the latency to improve the
> > > performance in checkpoint period;
> > >
> > > 3. Is smp_wmb() able to be safely disabled during the checkpoint period
> >
> > Well you'd have to understand where it's used; but for example, when taking
> > a checkpoint you'd want to be sure that the checkpoint data contained
> > a consistent copy of the last write data from all of the vCPUs; so I think
> > a wmb would be needed to make sure it's consistent.
> >
> > I'm surprised that the smp_wmb is such a big chunk of your total checkpoint
> > time, and that it's quite so long.
> > Are the vCPUs idle or are they busy - does it make difference?
> >
> > Dave
> >
> > > Really appreciate your help with my problems and hope to receive your
> > > feedback soon.
> > >
> > >
> > > Thanks again for your contribution to QEMU and it is such a masterpiece.
> >
> > Dave
> >
> > >
> > >
> > >
> > > Thanks and best regards,
> > >
> > > Niko Jiasheng Feng
> > >
> > > University of Hong Kong
> > >
> > > --
> > > *Niko Jiasheng *
> > > *Feng **Computer Science(General Stream), Faculty of Engineering, The
> > > University of Hong Kong*
> > > Contact:  （852）97908620
> > > Address: Pokfulam Road, The University of Hong Kong
> > > Email:      address@hidden / address@hidden
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >
> 
> 
> 
> -- 
> *Niko Jiasheng *
> *Feng **Computer Science(General Stream), Faculty of Engineering, The
> University of Hong Kong*
> Contact:  （852）97908620
> Address: Pokfulam Road, The University of Hong Kong
> Email:      address@hidden / address@hidden
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK