RE: [PATCH 02/10] Fix the qemu crash when guest shutdown during checkpoint

[Rao, Lei]

If user executes the shutdown normally and QEMU crashes, I think this is 
unacceptable.
Since we can avoid this situation, why not do it?

Thanks,
Lei.

-----Original Message-----
From: Lukas Straub <lukasstraub2@web.de> 
Sent: Sunday, February 14, 2021 7:46 PM
To: Rao, Lei <lei.rao@intel.com>
Cc: Zhang, Chen <chen.zhang@intel.com>; lizhijian@cn.fujitsu.com; 
jasowang@redhat.com; zhang.zhanghailiang@huawei.com; quintela@redhat.com; 
dgilbert@redhat.com; qemu-devel@nongnu.org
Subject: Re: [PATCH 02/10] Fix the qemu crash when guest shutdown during 
checkpoint

On Fri, 29 Jan 2021 02:57:57 +0000
"Rao, Lei" <lei.rao@intel.com> wrote:

> The state will be set RUN_STATE_COLO in colo_do_checkpoint_transaction(). If 
> the guest executes power off or shutdown at this time and the QEMU main 
> thread will call vm_shutdown(), it will set the state to RUN_STATE_SHUTDOWN.
> The state switch from RUN_STATE_COLO to RUN_STATE_SHUTDOWN is not defined in 
> runstate_transitions_def. this will cause QEMU crash. Although this is small 
> probability, it may still happen.

This patch fixes the 'colo' -> 'shutdown' transition. AFAIK then 
colo_do_checkpoint_transaction will call vm_start() again, which does 
'shutdown' -> 'running' and (rightfully) crashes. So I think it is better to 
crash here too.

>  By the way. Do you have any comments about other patches?
> Thanks,
> Lei.
> 
> -----Original Message-----
> From: Lukas Straub <lukasstraub2@web.de>
> Sent: Thursday, January 28, 2021 2:24 AM
> To: Rao, Lei <lei.rao@intel.com>
> Cc: Zhang, Chen <chen.zhang@intel.com>; lizhijian@cn.fujitsu.com; 
> jasowang@redhat.com; zhang.zhanghailiang@huawei.com; 
> quintela@redhat.com; dgilbert@redhat.com; qemu-devel@nongnu.org
> Subject: Re: [PATCH 02/10] Fix the qemu crash when guest shutdown 
> during checkpoint
> 
> On Thu, 21 Jan 2021 01:48:31 +0000
> "Rao, Lei" <lei.rao@intel.com> wrote:
> 
> > The Primary VM can be shut down when it is in COLO state, which may trigger 
> > this bug.  
> 
> Do you have a backtrace for this bug?
> 
> > About 'shutdown' -> 'colo' -> 'running', I think you are right, I did have 
> > the problems you said. For 'shutdown'->'colo', The fixed 
> > patch(5647051f432b7c9b57525470b0a79a31339062d2) have been merged.
> > Recently, I found another bug as follows in the test.
> >     qemu-system-x86_64: invalid runstate transition: 'shutdown' -> 'running'
> >             Aborted (core dumped)
> > The gdb bt as following:
> >     #0  __GI_raise (sig=sig@entry=6) at 
> > ../sysdeps/unix/sysv/linux/raise.c:50
> >     #1  0x00007faa3d613859 in __GI_abort () at abort.c:79
> >     #2  0x000055c5a21268fd in runstate_set (new_state=RUN_STATE_RUNNING) at 
> > vl.c:723
> >     #3  0x000055c5a1f8cae4 in vm_prepare_start () at 
> > /home/workspace/colo-qemu/cpus.c:2206
> >     #4  0x000055c5a1f8cb1b in vm_start () at 
> > /home/workspace/colo-qemu/cpus.c:2213
> >     #5  0x000055c5a2332bba in migration_iteration_finish (s=0x55c5a4658810) 
> > at migration/migration.c:3376
> >     #6  0x000055c5a2332f3b in migration_thread (opaque=0x55c5a4658810) at 
> > migration/migration.c:3527
> >     #7  0x000055c5a251d68a in qemu_thread_start (args=0x55c5a5491a70) at 
> > util/qemu-thread-posix.c:519
> >     #8  0x00007faa3d7e9609 in start_thread (arg=<optimized out>) at 
> > pthread_create.c:477
> >     #9  0x00007faa3d710293 in clone () at
> > ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> > 
> > For the bug, I made the following changes:
> >     @@ -3379,7 +3379,9 @@ static void 
> > migration_iteration_finish(MigrationState *s)
> >      case MIGRATION_STATUS_CANCELLED:
> >      case MIGRATION_STATUS_CANCELLING:
> >          if (s->vm_was_running) {
> > -            vm_start();
> > +            if (!runstate_check(RUN_STATE_SHUTDOWN)) {
> > +                vm_start();
> > +            }
> >          } else {
> >              if (runstate_check(RUN_STATE_FINISH_MIGRATE)) {
> >                  runstate_set(RUN_STATE_POSTMIGRATE);
> >                              
> > I will send the patch to community after more test.
> > 
> > Thanks,
> > Lei.
> > 
> > -----Original Message-----
> > From: Lukas Straub <lukasstraub2@web.de>
> > Sent: Thursday, January 21, 2021 3:13 AM
> > To: Rao, Lei <lei.rao@intel.com>
> > Cc: Zhang, Chen <chen.zhang@intel.com>; lizhijian@cn.fujitsu.com; 
> > jasowang@redhat.com; zhang.zhanghailiang@huawei.com; 
> > quintela@redhat.com; dgilbert@redhat.com; qemu-devel@nongnu.org
> > Subject: Re: [PATCH 02/10] Fix the qemu crash when guest shutdown 
> > during checkpoint
> > 
> > On Wed, 13 Jan 2021 10:46:27 +0800
> > leirao <lei.rao@intel.com> wrote:
> >   
> > > From: "Rao, Lei" <lei.rao@intel.com>
> > > 
> > > This patch fixes the following:
> > >     qemu-system-x86_64: invalid runstate transition: 'colo' ->'shutdown'
> > >     Aborted (core dumped)
> > > 
> > > Signed-off-by: Lei Rao <lei.rao@intel.com>
> > 
> > I wonder how that is possible, since the VM is stopped during 'colo' state.
> > 
> > Unrelated to this patch, I think this area needs some work since the 
> > following unintended runstate transition is possible:
> > 'shutdown' -> 'colo' -> 'running'.
> >   
> > > ---
> > >  softmmu/runstate.c | 1 +
> > >  1 file changed, 1 insertion(+)
> > > 
> > > diff --git a/softmmu/runstate.c b/softmmu/runstate.c index 
> > > 636aab0..455ad0d 100644
> > > --- a/softmmu/runstate.c
> > > +++ b/softmmu/runstate.c
> > > @@ -125,6 +125,7 @@ static const RunStateTransition 
> > > runstate_transitions_def[] = {
> > >      { RUN_STATE_RESTORE_VM, RUN_STATE_PRELAUNCH },
> > >  
> > >      { RUN_STATE_COLO, RUN_STATE_RUNNING },
> > > +    { RUN_STATE_COLO, RUN_STATE_SHUTDOWN},
> > >  
> > >      { RUN_STATE_RUNNING, RUN_STATE_DEBUG },
> > >      { RUN_STATE_RUNNING, RUN_STATE_INTERNAL_ERROR },
> > 
> > 
> >   
> 
> 
> 



--