On Mon, Dec 3, 2018 at 4:44 PM Rob Bradford <address@hidden> wrote:
Hi Stefano, thanks for capturing all these numbers,
On Mon, 2018-12-03 at 15:27 +0100, Stefano Garzarella wrote:
Hi Rob,
I continued to investigate the boot time, and as you suggested I
looked also at qemu-lite 2.11.2
(https://github.com/kata-containers/qemu) and NEMU "virt" machine. I
did the following tests using the Kata kernel configuration
(
https://github.com/kata-containers/packaging/blob/master/kernel/configs/x86_64_kata_kvm_4.14.x
)
To compare the results with qemu-lite direct kernel load, I added
another tracepoint:
- linux_start_kernel: first entry of the Linux kernel
(start_kernel())
Great, do you have a set of patches available that all these trace
points. It would be great for reproduction.
For sure! I'm attaching a set of patches for qboot, seabios, ovmf,
nemu/qemu/qemu-lite and linux 4.14 whit the tracepoints.
I'm also sharing a python script that I'm using with perf to extract
the numbers in this way:
$ perf record -a -e kvm:kvm_entry -e kvm:kvm_pio -e
sched:sched_process_exec -o /tmp/qemu_perf.data &
$ # start qemu/nemu multiple times
$ killall perf
$ perf script -s qemu-perf-script.py -i /tmp/qemu_perf.data
As you can see, NEMU is faster to jump to the kernel
(linux_start_kernel) than qemu-lite when uses qboot or seabios with
virt support, but the time to the user space is strangely high, maybe
the kernel configuration that I used is not the best one.
Do you suggest another kernel configuration?
This looks very bad. This isn't the kernel configuration we normally
test with in our automated test system but is definitely one we support
as part of our partnernship with the Kata team. It's a high priority
for me to try and investigate that. Have you saved the kernel messages
as they might be helpful?
Yes, I'm attaching the dmesg output with nemu and qemu.
Anyway, I obtained the best boot time with qemu-lite and direct
kernel
load (vmlinux ELF image). I think because the kernel was not
compressed. Indeed, looking to the others test, the kernel
decompression (bzImage) takes about 80 ms (linux_start_kernel -
linux_start_boot). (I'll investigate better)
Yup being able to load an uncompressed kernel is one of the big
advantages of qemu-lite. I wonder if we could bring that feature into
qemu itself to supplement the existing firmware based kernel loading.
I think so, I'll try to understand if we can merge the qemu-lite
direct kernel loading in qemu.