Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#2] Dissecting QEMU Into Three Main Parts

[Aleksandar Markovic]

уто, 30. јун 2020. у 06:34 Lukáš Doktor <ldoktor@redhat.com> је написао/ла:
>
> Dne 29. 06. 20 v 12:25 Ahmed Karaman napsal(a):
> > Hi,
> >
> > The second report of the TCG Continuous Benchmarking series builds
> > upon the QEMU performance metrics calculated in the previous report.
> > This report presents a method to dissect the number of instructions
> > executed by a QEMU invocation into three main phases:
> > - Code Generation
> > - JIT Execution
> > - Helpers Execution
> > It devises a Python script that automates this process.
> >
> > After that, the report presents an experiment for comparing the
> > output of running the script on 17 different targets. Many conclusions
> > can be drawn from the results and two of them are discussed in the
> > analysis section.
> >
> > Report link:
> > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Dissecting-QEMU-Into-Three-Main-Parts/
> >
> > Previous reports:
> > Report 1 - Measuring Basic Performance Metrics of QEMU:
> > https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06692.html
> >
> > Best regards,
> > Ahmed Karaman
>
> Hello Ahmed,
>
> very nice reading, both reports so far. One thing that could be better 
> displayed is the system you used this to generate. This would come handy 
> especially later when you move from examples to actual reports. I think it'd 
> make sense to add a section with a clear definition of the machine as well as 
> the operation system, qemu version and eventually other deps (like compiler, 
> flags, ...). For this report something like:
>
> architecture: x86_64
> cpu_codename: Kaby Lake
> cpu: i7-8650U
> ram: 32GB DDR4
> os: Fedora 32
> qemu: 470dd165d152ff7ceac61c7b71c2b89220b3aad7
> compiler: gcc-10.1.1-1.fc32.x86_64
> flags: 
> --target-list="x86_64-softmmu,ppc64-softmmu,aarch64-softmmu,s390x-softmmu,riscv64-softmmu"
>  --disable-werror --disable-sparse --enable-sdl --enable-kvm  
> --enable-vhost-net --enable-vhost-net --enable-attr  --enable-kvm  
> --enable-fdt   --enable-vnc --enable-seccomp 
> --block-drv-rw-whitelist="vmdk,null-aio,quorum,null-co,blkverify,file,nbd,raw,blkdebug,host_device,qed,nbd,iscsi,gluster,rbd,qcow2,throttle,copy-on-read"
>  --python=/usr/bin/python3 --enable-linux-io-uring
>
> would do. Maybe it'd be even a good idea to create a script to report this 
> basic set of information and add it after each of the perf scripts so people 
> don't forget to double-check the conditions, but others might disagree so 
> take this only as a suggestion.
>

I just want to follow up on this observation here, and not related to
Ahmed's report at all.

We often receive bug reports of the following style: "I have Debian
10.2 system and mips emulation misbehaves". As you may imagine, I
assign the bug to myself, install Debian 10.2 system on my
experimental box, and mips emulation works like charm.
<banging-head-against-the-wall-emoji> Obviously, I need more info on
the submitter's system.

After all these years, we don't have (or at least I don't know about
it) a script that we could give the submitter, that picks up various
aspects of his system. This script, since it is not "for presentation"
could be even far more aggressive in picking ups system information
that what Lukas mentioned above. It could collect the output of
various relevant commands, and yip it in a single file. We should have
"get_system_info.py" in our scripts directory!

Sincerely,
Aleksandar


> Regards,
> Lukáš
>
> PS: Automated cpu codenames, hosts OSes and such could be tricky, but one can 
> use other libraries or just best-effort-approach with fallback to "unknown" 
> to let people filling it manually or adding their branch to your script.
>
> Regards,
> Lukáš
>