Re: [Qemu-devel] QEMU/NEMU boot time with several x86 firmwares

[Stefano Garzarella]

On Mon, Dec 3, 2018 at 4:44 PM Rob Bradford <address@hidden> wrote:
>
> Hi Stefano, thanks for capturing all these numbers,
>
> On Mon, 2018-12-03 at 15:27 +0100, Stefano Garzarella wrote:
> > Hi Rob,
> > I continued to investigate the boot time, and as you suggested I
> > looked also at qemu-lite 2.11.2
> > (https://github.com/kata-containers/qemu) and NEMU "virt" machine. I
> > did the following tests using the Kata kernel configuration
> > (
> > https://github.com/kata-containers/packaging/blob/master/kernel/configs/x86_64_kata_kvm_4.14.x
> > )
> >
> > To compare the results with qemu-lite direct kernel load, I added
> > another tracepoint:
> > - linux_start_kernel: first entry of the Linux kernel
> > (start_kernel())
> >
>
> Great, do you have a set of patches available that all these trace
> points. It would be great for reproduction.

For sure! I'm attaching a set of patches for qboot, seabios, ovmf,
nemu/qemu/qemu-lite and linux 4.14 whit the tracepoints.
I'm also sharing a python script that I'm using with perf to extract
the numbers in this way:

$ perf record -a -e kvm:kvm_entry -e kvm:kvm_pio -e
sched:sched_process_exec -o /tmp/qemu_perf.data &
$ # start qemu/nemu multiple times
$ killall perf
$ perf script -s qemu-perf-script.py -i /tmp/qemu_perf.data

>
> > As you can see, NEMU is faster to jump to the kernel
> > (linux_start_kernel) than qemu-lite when uses qboot or seabios with
> > virt support, but the time to the user space is strangely high, maybe
> > the kernel configuration that I used is not the best one.
> > Do you suggest another kernel configuration?
> >
>
> This looks very bad. This isn't the kernel configuration we normally
> test with in our automated test system but is definitely one we support
> as part of our partnernship with the Kata team. It's a high priority
> for me to try and investigate that. Have you saved the kernel messages
> as they might be helpful?

Yes, I'm attaching the dmesg output with nemu and qemu.

>
> > Anyway, I obtained the best boot time with qemu-lite and direct
> > kernel
> > load (vmlinux ELF image). I think because the kernel was not
> > compressed. Indeed, looking to the others test, the kernel
> > decompression (bzImage) takes about 80 ms (linux_start_kernel -
> > linux_start_boot). (I'll investigate better)
> >
>
> Yup being able to load an uncompressed kernel is one of the big
> advantages of qemu-lite. I wonder if we could bring that feature into
> qemu itself to supplement the existing firmware based kernel loading.

I think so, I'll try to understand if we can merge the qemu-lite
direct kernel loading in qemu.


Thanks,
Stefano


-- 
Stefano Garzarella
Red Hat

qemu_dmesg.txt
Description: Text document

nemu_dmesg.txt
Description: Text document

qemu-perf-script.py
Description: Text Data

nemu-bench.patch
Description: Text Data

seabios-bench.patch
Description: Text Data

qboot-bench.patch
Description: Text Data

linux-v4.14.67-bench.patch
Description: Text Data