[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization
From: |
Simon Kaegi |
Subject: |
[Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization |
Date: |
Fri, 24 Jul 2020 04:36:13 -0000 |
I believe the VSI itself is QEMU based but don't know the version or
details but suspect it's 4.1 based. We compile our own QEMU version for
use with Kata and that's where we're now using 5.1.0-rc1 with the above
commit reverted.
Host Kernel is ... 4.15.0-101-generic if that helps
re: cpu -- four of these...
```
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
microcode : 0x1
cpu MHz : 2095.148
cache size : 16384 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm
constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx
ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault
invpcid_single pti ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid
fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap
xsaveopt arat md_clear
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
swapgs taa itlb_multihit
bogomips : 4190.29
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
```
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1888601
Title:
QEMU v5.1.0-rc0/rc1 hang with nested virtualization
Status in QEMU:
New
Bug description:
We're running Kata Containers using QEMU and with v5.1.0rc0 and rc1
have noticed a problem at startup where QEMu appears to hang. We are
not seeing this problem on our bare metal nodes and only on a VSI that
supports nested virtualization.
We unfortunately see nothing at all in the QEMU logs to help
understand the problem and a hung process is just a guess at this
point.
Using git bisect we first see the problem with...
---
f19bcdfedd53ee93412d535a842a89fa27cae7f2 is the first bad commit
commit f19bcdfedd53ee93412d535a842a89fa27cae7f2
Author: Jason Wang <jasowang@redhat.com>
Date: Wed Jul 1 22:55:28 2020 +0800
virtio-pci: implement queue_enabled method
With version 1, we can detect whether a queue is enabled via
queue_enabled.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Cindy Lu <lulu@redhat.com>
Message-Id: <20200701145538.22333-5-lulu@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
hw/virtio/virtio-pci.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
---
Reverting this commit (on top of 5.1.0-rc1) seems to work and prevent
the hanging.
---
Here's how kata ends up launching qemu in our environment --
/opt/kata/bin/qemu-system-x86_64 -name
sandbox-849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f -uuid
6bec458e-1da7-4847-a5d7-5ab31d4d2465 -machine pc,accel=kvm,kernel_irqchip -cpu
host,pmu=off -qmp
unix:/run/vc/vm/849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f/qmp.sock,server,nowait
-m 4096M,slots=10,maxmem=30978M -device
pci-bridge,bus=pci.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile=
-device virtio-serial-pci,disable-modern=true,id=serial0,romfile= -device
virtconsole,chardev=charconsole0,id=console0 -chardev
socket,id=charconsole0,path=/run/vc/vm/849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f/console.sock,server,nowait
-device virtio-scsi-pci,id=scsi0,disable-modern=true,romfile= -object
rng-random,id=rng0,filename=/dev/urandom -device
virtio-rng-pci,rng=rng0,romfile= -device
virtserialport,chardev=charch0,id=channel0,name=agent.channel.0 -chardev
socket,id=charch0,path=/run/vc/vm/849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f/kata.sock,server,nowait
-chardev
socket,id=char-396c5c3e19e29353,path=/run/vc/vm/849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f/vhost-fs.sock
-device
vhost-user-fs-pci,chardev=char-396c5c3e19e29353,tag=kataShared,romfile= -netdev
tap,id=network-0,vhost=on,vhostfds=3:4,fds=5:6 -device
driver=virtio-net-pci,netdev=network-0,mac=52:ac:2d:02:1f:6f,disable-modern=true,mq=on,vectors=6,romfile=
-global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults
-nographic -daemonize -object
memory-backend-file,id=dimm1,size=4096M,mem-path=/dev/shm,share=on -numa
node,memdev=dimm1 -kernel /opt/kata/share/kata-containers/vmlinuz-5.7.9-74
-initrd
/opt/kata/share/kata-containers/kata-containers-initrd_alpine_1.11.2-6_agent.initrd
-append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1
i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0
console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug
panic=1 nr_cpus=4 agent.use_vsock=false scsi_mod.scan=none
init=/usr/bin/kata-agent -pidfile
/run/vc/vm/849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f/pid
-D
/run/vc/vm/849df14c6065931adedb9d18bc9260a6d896f1814a8c5cfa239865772f1b7a5f/qemu.log
-smp 2,cores=1,threads=1,sockets=4,maxcpus=4
---
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1888601/+subscriptions
- [Bug 1888601] [NEW] QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/22
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/23
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Dr. David Alan Gilbert, 2020/07/23
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization,
Simon Kaegi <=
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Jason Wang, 2020/07/24
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Dr. David Alan Gilbert, 2020/07/24
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/24
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/24
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Jason Wang, 2020/07/26
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/27
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/29
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/29
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Jason Wang, 2020/07/29
- [Bug 1888601] Re: QEMU v5.1.0-rc0/rc1 hang with nested virtualization, Simon Kaegi, 2020/07/30