Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Laszlo Ersek]

On 08/30/13 05:28, Gary Ching-Pang Lin wrote:
> On Fri, Aug 30, 2013 at 02:04:40AM +1000, Bruce Rogers wrote:
>>  >>> On 8/29/2013 at 02:23 AM, Gary Ching-Pang Lin <address@hidden> wrote: 
>>> On Wed, Aug 28, 2013 at 02:55:26PM +0200, Andreas Färber wrote:
>>>> Am 28.08.2013 14:10, schrieb Laszlo Ersek:
>>>>> On 08/28/13 13:49, Andreas Färber wrote:
>>>>>> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
>>>>>>> (qemu-devel CC'd)
>>>>>>>
>>>>>>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I recently updated qemu to 1.6.0 and found OVMF just showed a blank
>>>>>>>> screen when kvm was enabled. I tried to dump OVMF log with the
>>>>>>>> following commond but nothing was stored in debug.log.
>>>>>>>>
>>>>>>>> qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon 
>>>>>>>> file:debug.log -global 
>>> isa-debugcon.iobase=0x402
>>>>>>>>
>>>>>>>> The kvm trace was recorded with "trace-cmd record -b 20000 -e kvm"
>>>>>>>> and uploaded to the following link:
>>>>>>>>
>>> https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
>>>>>>>>
>>>>>>>> I found a similar case with kernel < 3.9, but I already upgraded linux
>>>>>>>> kernel to 3.10.5, so this may be another bug.
>>>>>>>
>>>>>>> Well, the usual first response in cases like this is...
>>>>>>>
>>>>>>> Can you bisect qemu? :)
>>>>>>
>>>>>> We had a similar report:
>>>>>> https://bugzilla.novell.com/show_bug.cgi?id=835895
>>>>>
>>>>> Well that's sorta the same report, considering you and Gary both work
>>>>> for SUSE, and the Novell BZ seems to imply the build in question was 
>>> Gary's:
>>>>>
>>>>>> qemu 1.6.0 fails to run the tianocore firmware
>>>>>> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu
>>>>>         ^^^^^^^^
>>>>>> versions:
>>>>>
>>>>> :)
>>>>
>>>> Different reporters, so who knows if the setups are the same. ;)
>>>>
>>>>>> git-bisect said:
>>>>>> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
>>>>>> commit 235e8982ad393e5611cb892df54881c872eea9e1
>>>>>> Author: Jordan Justen <address@hidden>
>>>>>> Date:   Wed May 29 01:27:26 2013 -0700
>>>>>>
>>>>>>     kvm: support using KVM_MEM_READONLY flag for regions
>>>>>>
>>>>>>     For readonly memory regions and rom devices in romd_mode,
>>>>>>     we make use of the KVM_MEM_READONLY. A slot that uses
>>>>>>     KVM_MEM_READONLY can be read from and code can execute from the
>>>>>>     region, but writes will exit to qemu.
>>>>>>
>>>>>>     For rom devices with !romd_mode, we force the slot to be
>>>>>>     removed so reads or writes to the region will exit to qemu.
>>>>>>     (Note that a memory region in this state is not executable
>>>>>>     within kvm.)
>>>>>>
>>>>>>     v7:
>>>>>>      * Update for readable => romd_mode rename (5f9a5ea1)
>>>>>>
>>>>>>     Signed-off-by: Jordan Justen <address@hidden>
>>>>>>     Reviewed-by: Xiao Guangrong <address@hidden> (v4)
>>>>>>     Reviewed-by: Paolo Bonzini <address@hidden> (v5)
>>>>>>     Message-id: address@hidden
>>>>>>     Signed-off-by: Anthony Liguori <address@hidden>
>>>>>>
>>>>>>
>>>>>> Any hints or patches welcome. :)
>>>>>
>>>>> Hm. LP 1212402 <https://bugs.launchpad.net/qemu/+bug/1212402> probably
>>>>> concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
>>>>> likely not revelant here.
>>>>>
>>>>>
>>>>> Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?
>>>>>
>>>>> Author: Jordan Justen <address@hidden>
>>>>> Date:   Thu Jul 18 22:51:27 2013 +0000
>>>>>
>>>>>     OvmfPkg/Sec: Build identity mapped pages in RAM for X64
>>>>>
>>>>>     This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.
>>>>>
>>>>>     Previously we would run using page tables built into the
>>>>>     firmware device.
>>>>>
>>>>>     If a flash memory is available, it is unsafe for the page
>>>>>     tables to be stored in memory since the processor may try
>>>>>     to write to the page table data structures.
>>>>>
>>>>>     Additionally, when KVM ROM support is enabled for the
>>>>>     firmware device, then PEI fails to boot when the page
>>>>>     tables are in the firmware device.
>>>>
>>>> https://build.opensuse.org/package/show/Virtualization/OVMF
>>>> is at r14547 and that one works for me.
>>>>
>>>> Gary/Ludwig, can you confirm that this is resolved?
>>>>
>>> I still got the black screen even with OVMF r14547 or r14608. I also
>>> tried kernel 3.10.9 and 3.11-rc7, but no luck.
>>>
>>> Reverting the KVM_MEM_READONLY patch works for me, but the OVMF patch
>>> somehow didn't work.
>>>
>>> Thanks,
>>>
>>> Gary Lin
>>
>> I tried this out, and I get the black screen as well when ept=n, but it boots
>> successfully to the efi shell when ept=y.
>>
>> Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
>> failing machine?
>>
> I think this is the root cause. My machine doesn't support ept.

Disclaimer: I don't know what I'm talking about.

So, Jordan's patch for OVMF (SVN r14494) builds the page tables (and
finally writes the root to CR3) in a phase when paging is not enabled
yet in the VM.

As far as I know, if you don't have EPT (nested paging) then the
hypervisor must maintain shadow page tables -- it must catch the guest's
writes to the guest's own paging structures (eg. by write protecting
them and handling the faults), validating and mirroring the guest's
changes to the hypervisor's shadow tables.

Again, I have no clue, but if the guest hasn't even enabled paging yet,
then the hypervisor (without EPT?) might have no idea that what the
guest is writing to memory are its pagetables-to-be. The first notice
the hypervisor might take is the store to CR3. At which point (or maybe
even later, when paging is enabled?) the hypervisor would have to walk
the guest's tables all at once, and build the shadow tables "in batch".

(The above is probably exceedingly lame, sorry...)

Laszlo