qemu-ppc
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-ppc] KVM internal error with kvm-pr on ppc64le


From: Loic Devulder
Subject: [Qemu-ppc] KVM internal error with kvm-pr on ppc64le
Date: Wed, 30 May 2018 14:59:53 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0

Hi guys!

I tried to start a Linux CD (I tried openSUSE Leap 42.3) to install my ppc64le 
VM with kvm-pr (yes
kvm-pr!) on a Power8 host and I have some issues...

My host system is an openSUSE Tumbleweed 20180525 with qemu 2.11 and kernel 
4.16.11.

I tried different version of qemu without any success but I tried with openSUSE 
Leap 42.3 as a host
(and qemu 2.11 and 2.12) with success. Kernel is 4.4 on Leap 42.3, and if I 
update the kernel to a
more recent version I have the same issue! So it seems to be a kvm problem, not 
qemu.

I also tried without KVM and it works (but it is of course very slow!).

I tried to understand why and it seems to be related to the virtio-vga. Here 
the different tries I
made with openSUSE Tumbleweed 20180525 as host.

- With only virtio-vga and boot/messages on VGA console:
/usr/bin/qemu-system-ppc64 -m 1024 -cpu host -device virtio-scsi-pci,id=scsi0 
-device
scsi-cd,drive=cd0,bus=scsi0.0 -drive
media=cdrom,if=none,id=cd0,format=raw,file=/opt/openSUSE-Leap-42.3-NET-ppc64le-Build0130-Media.iso
-smp 1,threads=1 -enable-kvm -device nec-usb-xhci -device usb-tablet -device 
usb-kbd -serial
stdio:mon -device virtio-vga -nodefaults

Trying to load:  from: /address@hidden/address@hidden/address@hidden ...   
Successfully loaded
SUSE Linux
#1 SMP Fri Jun 3KVM internal error. Suberror: 61568
NIP c0000000005359f4   LR c0000000005356cc CTR 0000000000000032 XER 
0000000000000000 CPU#0
MSR b000000000009033 HID0 0000000000000000  HF 8000000000000000 iidx 3 didx 3
TB 00000000 00000000 DECR 00000000
GPR00 0000000000006400 c00000003e49f440 c000000000f3ba00 0000000000000190
GPR04 d0000800803ce800 0000000000000000 0000000000000000 0000000000000000
GPR08 0000000000000040 d0000800803ce800 0000000000000032 ffffffffffffffff
GPR12 c000000000535950 c00000000fa00000 c00000000000bf38 0000000000000000
GPR16 0000000000000064 0000000000000025 0000000000000064 c0000000012d5a98
GPR20 0000000000000000 0000000000000320 0000000000000000 0000000000000000
GPR24 0000000000000006 0000000000000008 0000000000000000 0000000000006400
GPR28 c000000000535950 c00000003e719800 0000000000000000 d0000800803ce800
CR 44022084  [ G  G  -  E  E  -  L  G  ]             RES ffffffffffffffff
FPR00 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPSCR 0000000000000000
 SRR0 c0000000005359f4  SRR1 b000000000009033    PVR 00000000004b0201 VRSAVE 
0000000000000000
SPRG0 0000000000000000 SPRG1 c00000000fa00000  SPRG2 c00000000fa00000  SPRG3 
0000000000000000
SPRG4 0000000000000000 SPRG5 0000000000000000  SPRG6 0000000000000000  SPRG7 
0000000000000000
HSRR0 0000000000000000 HSRR1 0000000000000000
 CFAR 0000000000000000
 LPCR 0000000002000000
 SDR1 0000000000000000   DAR d0000800803ce800  DSISR 0000000042000000

I have the "Loading kernel..." to "Booting Linux via __start() ..." on the VGA 
console and after the
crash on the serial console. "Suberror: 61568" is weird...

Same if I boot with "-vga std" or "-vga virtio" instead of "-device virtio-vga" 
(I know that "-vga
virtio" and "-device virtio-vga" are the same).

With the help of Alexander Graf (thanks!) I was able to have some debug 
informations:
qemu-system-ppc-9944  [006] .... 83419.417302: kvm_book3s_reenter: reentry r=1 
| pc=0xc000000000089930
 qemu-system-ppc-9944  [006] .... 83419.417304: kvm_exit: exit=PROGRAM | 
pc=0xc000000000009958 |
msr=0xb000000000001031 | dar=0x7fff67356200 | srr1=0x800000000004d033 | 
last_inst=0x4c000024
 qemu-system-ppc-9944  [006] .... 83419.417305: kvm_ppc_instr: inst 1275068452 
pc 0xc0000000005359f4
emulate 0
 qemu-system-ppc-9944  [006] .... 83419.417305: kvm_book3s_reenter: reentry r=1 
| pc=0xc0000000005359f4
 qemu-system-ppc-9944  [006] .... 83419.417307: kvm_exit: exit=DATA_SEGMENT | 
pc=0xc0000000005359f4
| msr=0xb000000000009033 | dar=0xd0000800803ce800 | srr1=0x800000000000d033 | 
last_inst=0x4c000024
 qemu-system-ppc-9944  [006] .... 83419.417307: kvm_book3s_slb_found: 
18000033deaf8008 -> 61c0f87e5e
 qemu-system-ppc-9944  [006] .... 83419.417308: kvm_book3s_slbmte: 
61c0f87e5e990, d000080088000003
 qemu-system-ppc-9944  [006] .... 83419.417308: kvm_book3s_reenter: reentry r=0 
| pc=0xc0000000005359f4
 qemu-system-ppc-9944  [006] .... 83419.417310: kvm_exit: exit=DATA_STORAGE | 
pc=0xc0000000005359f4
| msr=0xb000000000009033 | dar=0xd0000800803ce800 | srr1=0x800000000000d033 | 
last_inst=0xf8c90000
 qemu-system-ppc-9944  [006] .... 83419.417315: kvm_book3s_slb_found: 
18000033deaf8008 -> 61c0f87e5e
 qemu-system-ppc-9944  [006] .... 83419.417317: kvm_book3s_reenter: reentry r=2 
| pc=0xc0000000005359f4
 qemu-system-ppc-9944  [006] .... 83419.417319: kvm_userspace_exit: reason 
KVM_EXIT_INTERNAL_ERROR (17)
qemu-system-ppc-9941  [004] .... 83419.447328: kvm_book3s_mmu_flush: Flush 5473 
pPTEs: 200080000000
- 200081000000
           gdbus-9946  [004] .... 83465.282674: kvm_book3s_mmu_flush: Flush 
5473 PTEs: 0 - 0
           gdbus-9946  [004] .... 83465.282684: kvm_book3s_mmu_invalidate: 
Flush: hva=61c1b7cf37f7f0
pfn=7677e ea=c00000002f7f0000 vp=ea1b217002f7f0 ra=2f7f0000 [6]

- With only virtio-vga and boot/messages on hvterm console (I removed the USB 
part to force the
console output):
/usr/bin/qemu-system-ppc64 -serial mon:stdio -m 1024 -cpu host -device 
virtio-scsi-pci,id=scsi0
-device scsi-cd,drive=cd0,bus=scsi0.0 -drive
media=cdrom,if=none,id=cd0,format=raw,file=/opt/openSUSE-Leap-42.3-NET-ppc64le-Build0130-Media.iso
-smp 1,threads=1 -enable-kvm -device virtio-vga -nodefaults

Loading kernel ...
Loading initial ramdisk ...
OF stdout device is: /vdevice/address@hidden
Preparing to boot Linux version 4.4.74-1-default (address@hidden) (gcc version 
4.8.5 (SUSE Linux) )
#1 SMP Fri Jun 30 19:11:19 UTC 2017 (f0a86d9)
Detected machine type: 0000000000000101
Max number of cores passed to firmware: 2048 (NR_CPUS = 2048)
Calling ibm,client-architecture-support... done
command line: BOOT_IMAGE=/boot/ppc64le/linux
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 0000000006380000
  alloc_top    : 0000000030000000
  alloc_top_hi : 0000000040000000
  rmo_top      : 0000000030000000
  ram_top      : 0000000040000000
found display   : /address@hidden/address@hidden, opening... done
instantiating rtas at 0x000000002fff0000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x0000000006390000 -> 0x0000000006390abe
Device tree struct  0x00000000063a0000 -> 0x00000000063b0000
Quiescing Open Firmware ...
Booting Linux via __start() ...
Loading kernel ...
Loading initial ramdisk ...
OF stdout device is: /vdevice/address@hidden
Preparing to boot Linux version 4.4.74-1-default (address@hidden) (gcc version 
4.8.5 (SUSE Linux) )
#1 SMP Fri Jun 30 19:11:19 UTC 2017 (f0a86d9)
Detected machine type: 0000000000000101
Max number of cores passed to firmware: 2048 (NR_CPUS = 2048)
Calling ibm,client-architecture-support... done
command line: BOOT_IMAGE=/boot/ppc64le/linux
[...]
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
Using unsupported 800x600 vga at 200080800000, depth=32, pitch=3200
KVM internal error. Suberror: 88
NIP c0000000005359f4   LR c0000000005356cc CTR 0000000000000032 XER 
0000000000000000 CPU#0
MSR b000000000009033 HID0 0000000000000000  HF 8000000000000000 iidx 3 didx 3
TB 00000000 00000000 DECR 00000000
GPR00 0000000000006400 c00000003e49f440 c000000000f3ba00 0000000000000190
GPR04 d0000800801ce800 0000000000000000 0000000000000000 0000000000000000
GPR08 0000000000000040 d0000800801ce800 0000000000000032 ffffffffffffffff
GPR12 c000000000535950 c00000000fa00000 c00000000000bf38 0000000000000000
GPR16 0000000000000064 0000000000000025 0000000000000064 c0000000012d5a98
GPR20 0000000000000000 0000000000000320 0000000000000000 0000000000000000
GPR24 0000000000000006 0000000000000008 0000000000000000 0000000000006400
GPR28 c000000000535950 c00000003e719800 0000000000000000 d0000800801ce800
CR 44022084  [ G  G  -  E  E  -  L  G  ]             RES ffffffffffffffff
FPR00 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPSCR 0000000000000000
 SRR0 c0000000005359f4  SRR1 b000000000009033    PVR 00000000004b0201 VRSAVE 
0000000000000000
SPRG0 0000000000000000 SPRG1 c00000000fa00000  SPRG2 c00000000fa00000  SPRG3 
0000000000000000
SPRG4 0000000000000000 SPRG5 0000000000000000  SPRG6 0000000000000000  SPRG7 
0000000000000000
HSRR0 0000000000000000 HSRR1 0000000000000000
 CFAR 0000000000000000
 LPCR 0000000002000000
 SDR1 0000000000000000   DAR d0000800801ce800  DSISR 0000000042000000

It's a little bit better, I think "Suberror: 88" is more comprehensible (at 
least I hope!).

Same if I boot with "-vga std" or "-vga virtio" instead of "-device virtio-vga".

Some debug informations:
qemu-system-ppc-10016 [004] .... 83794.482587: kvm_book3s_reenter: reentry r=1 
| pc=0xc000000000089930
 qemu-system-ppc-10016 [004] .... 83794.482588: kvm_exit: exit=PROGRAM | 
pc=0xc000000000009958 |
msr=0xb000000000001031 | dar=0x7fff3e357200 | srr1=0x800000000004d033 | 
last_inst=0x4c000024
 qemu-system-ppc-10016 [004] .... 83794.482589: kvm_ppc_instr: inst 1275068452 
pc 0xc0000000005359f4
emulate 0
 qemu-system-ppc-10016 [004] .... 83794.482590: kvm_book3s_reenter: reentry r=1 
| pc=0xc0000000005359f4
 qemu-system-ppc-10016 [004] .... 83794.482591: kvm_exit: exit=DATA_SEGMENT | 
pc=0xc0000000005359f4
| msr=0xb000000000009033 | dar=0xd0000800801ce800 | srr1=0x800000000000d033 | 
last_inst=0x4c000024
 qemu-system-ppc-10016 [004] .... 83794.482592: kvm_book3s_slb_found: 
18000033deaf8008 -> 61c1b7cf37
 qemu-system-ppc-10016 [004] .... 83794.482592: kvm_book3s_slbmte: 
61c1b7cf37990, d000080088000003
 qemu-system-ppc-10016 [004] .... 83794.482592: kvm_book3s_reenter: reentry r=0 
| pc=0xc0000000005359f4
 qemu-system-ppc-10016 [004] .... 83794.482594: kvm_exit: exit=DATA_STORAGE | 
pc=0xc0000000005359f4
| msr=0xb000000000009033 | dar=0xd0000800801ce800 | srr1=0x800000000000d033 | 
last_inst=0xf8c90000
 qemu-system-ppc-10016 [004] .... 83794.482599: kvm_book3s_slb_found: 
18000033deaf8008 -> 61c1b7cf37
 qemu-system-ppc-10016 [004] .... 83794.482602: kvm_book3s_reenter: reentry r=2 
| pc=0xc0000000005359f4
 qemu-system-ppc-10016 [004] .... 83794.482602: kvm_userspace_exit: reason 
KVM_EXIT_INTERNAL_ERROR (17)
 qemu-system-ppc-10013 [005] .... 83794.495264: kvm_book3s_mmu_flush: Flush 
5468 pPTEs: 200080000000
- 200081000000
 qemu-system-ppc-10013 [001] .... 83800.464923: kvm_book3s_mmu_flush: Flush 
5468 PTEs: 0 - 0
 qemu-system-ppc-10013 [001] .... 83800.464933: kvm_book3s_mmu_invalidate: 
Flush: hva=61c0392d85f7f0
pfn=75b7e ea=c00000002f7f0000 vp=ea1b217002f7f0 ra=2f7f0000 [6]
 qemu-system-ppc-10013 [001] .... 83800.464938: kvm_book3s_mmu_invalidate: 
Flush: hva=61c0392d85f7e0
pfn=75b7d ea=c00000002f7e0000 vp=ea1b217002f7e0 ra=2f7e0000 [6]

- With only virtio-gpu and boot/messages on hvterm/VGA console:
/usr/bin/qemu-system-ppc64 -serial mon:stdio -m 1024 -cpu host -device 
virtio-scsi-pci,id=scsi0
-device scsi-cd,drive=cd0,bus=scsi0.0 -drive
media=cdrom,if=none,id=cd0,format=raw,file=/opt/openSUSE-Leap-42.3-NET-ppc64le-Build0130-Media.iso
-smp 1,threads=1 -enable-kvm -device virtio-gpu -nodefaults

No problem, the kernel boot with default console on hvterm and some messages on 
VGA console. And the
kernel boot also if I set the console to VGA (console=tty on Linux).

- With only virtio-gpu-pci and boot/messages on hvterm:
/usr/bin/qemu-system-ppc64 -serial mon:stdio -m 1024 -cpu host -device 
virtio-scsi-pci,id=scsi0
-device scsi-cd,drive=cd0,bus=scsi0.0 -drive
media=cdrom,if=none,id=cd0,format=raw,file=/opt/openSUSE-Leap-42.3-NET-ppc64le-Build0130-Media.iso
-smp 1,threads=1 -enable-kvm -device virtio-gpu-pci -nodefaults

Same as above. It seems like if virtio-gpu and virtio-gpu-pci are the same.

QT_X11_NO_MITSHM=1 /usr/bin/qemu-system-ppc64 -serial mon:stdio -m 1024 -cpu 
host -device
virtio-scsi-pci,id=scsi0 -device scsi-cd,drive=cd0,bus=scsi0.0 -drive
media=cdrom,if=none,id=cd0,format=raw,file=/opt/openSUSE-Leap-42.3-NET-ppc64le-Build0130-Media.iso
-smp 1,threads=1 -enable-kvm -device nec-usb-xhci -device usb-tablet -device 
usb-kbd -device
virtio-vga -device virtio-gpu-pci -nodefaults

So, is anybody already saw this? May I did something wrong? How we can debug 
this? (And sorry for
this long email, I try to add all the needed information!).

-- 
Loic Devulder - Senior QA Engineer
Container & Storage Solutions Quality Assurance team (qa-css)
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB, 21284 (AG 
Nuernberg)



reply via email to

[Prev in Thread] Current Thread [Next in Thread]