Re: 5.1.0-rc1 regression: reset fails with kvm and -cpu host

[Dr. David Alan Gilbert]

* Vitaly Kuznetsov (vkuznets@redhat.com) wrote:
> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
> 
> > +Vitaly
> >
> > On 7/23/20 10:40 AM, Dr. David Alan Gilbert wrote:
> >> * Eduardo Habkost (ehabkost@redhat.com) wrote:
> >>> On Wed, Jul 22, 2020 at 04:47:32PM -0400, Eduardo Habkost wrote:
> >>>> On Wed, Jul 22, 2020 at 08:05:01PM +0200, Jan Kiszka wrote:
> >>>>> On 22.07.20 19:35, Eduardo Habkost wrote:
> >>>>>> Hi Jan,
> >>>>>>
> >>>>>> What was the last version where it worked for you?  Does using
> >>>>>> "-cpu host,-vmx" help?
> >>>>>
> >>>>> Yeah, -vmx does indeed help.
> >>>>>
> >>>>> I didn't have the time to bisect yet. Just check my reflog, picked
> >>>>> eb6490f544, and that works.
> >>>>
> >>>> Thanks!
> >>>>
> >>>> I could reproduce it locally[1], I will bisect it.
> >>>>
> >>>> The good news is that "-cpu host,+vmx" still works, on commit
> >>>> eb6490f544.
> >>>>
> >>>> [1] Linux 5.6.19-300.fc32.x86_64, Intel Core i7-8665U CPU.
> >>>
> >>> Bisected to:
> >>>
> >>> commit b16c0e20c74218f2d69710cedad11da7dd4d2190
> >>> Author: Paolo Bonzini <pbonzini@redhat.com>
> >>> Date:   Wed May 20 10:49:22 2020 -0400
> >>>
> >>>     KVM: add support for AMD nested live migration
> >>>
> >>>     Support for nested guest live migration is part of Linux 5.8, add the
> >>>     corresponding code to QEMU.  The migration format consists of a few
> >>>     flags, is an opaque 4k blob.
> >>>
> >>>     The blob is in VMCB format (the control area represents the L1 VMCB
> >>>     control fields, the save area represents the pre-vmentry state; KVM 
> >>> does
> >>>     not use the host save area since the AMD manual allows that) but QEMU
> >>>     does not really care about that.  However, the flags need to be
> >>>     copied to hflags/hflags2 and back.
> >>>
> >>>     In addition, support for retrieving and setting the AMD nested 
> >>> virtualization
> >>>     states allows the L1 guest to be reset while running a nested guest, 
> >>> but
> >>>     a small bug in CPU reset needs to be fixed for that to work.
> >>>
> >>>     Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> >> 
> >> Guesswork led me to try reverting the chunk in kvm_put_nested_state;
> >> without it the reset seems to work; I can't explain that code though.
> >> 
> 
> (sorry, missed the beginning of the discussion)
> 
> So one does:
> 
> (qemu) system_reset 
> 
> on Intel wiht '-cpu host' and the result is:
> 
> (qemu) KVM: entry failed, hardware error 0x80000021

Interesting; I hadn't seen that error - I just see a hard hung guest
rather than a reset one.

(i7-8650U laptop 5.7.9 fedora 32)

Dave

> If you're running a guest on an Intel machine without unrestricted mode
> support, the failure can be most likely due to the guest entering an invalid
> state for Intel VT. For example, the guest maybe running in big real mode
> which is not supported on less recent Intel processors.
> 
> EAX=00000064 EBX=91df5efe ECX=00000000 EDX=000003f8
> ESI=00000000 EDI=91ee32c0 EBP=90643260 ESP=00013c68
> EIP=906428e6 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0000 00000000 0000ffff 00009300
> CS =f000 ffff0000 0000ffff 00009b00
> SS =0000 00000000 0000ffff 00009300
> DS =0000 00000000 0000ffff 00009300
> FS =0000 00000000 0000ffff 00009300
> GS =0000 00000000 0000ffff 00009300
> LDT=0000 00000000 0000ffff 00008200
> TR =0000 00000000 0000ffff 00008b00
> GDT=     00000000 0000ffff
> IDT=     00000000 0000ffff
> CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 
> DR3=0000000000000000 
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> Code=?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? <??> ?? ?? 
> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? 
> ??
> 
> I can take a look (if no one beats me to it).
> 
> -- 
> Vitaly
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK