Re: [Qemu-discuss] problems achieving "fullspeed" system emulation (TCG, icount, sleep off, rtc is vm)

[Nutaro, James J.]

Hi Artem,


I've had to update the patch several times to follow the evolution of QEMU. If 
you are working with QEMU-2.12, try building with the patch available here:


https://web.ornl.gov/~nutarojj/adevs/


The adevs simulation package also contains two examples that exercise the QEMU 
synchronization feature.


Its not entirely clear from your post how you are coordinating your simulation 
clock with the QEMU clock. The intended use is for QEMU to participate in an 
overarching discrete event simulation. To summarize how the simulator/QEMU 
interaction is meant to work (with the new patch):


while(1)

{

   QEMU posts a time advance request

   QEMU waits for a time advance grant

   On receiving the grant, QEMU runs until its virtual clock equals the grant 
time

}


The request is issued to the overarching simulator, which then replies with the 
grant with the simulation clock has reached to requested time or some other 
event relevant to QEMU occurs in the simulation.


This may result in a simulation that is faster or slower than real-time, 
depending on the workload of QEMU and the other activities taking place in your 
simulation.


Let me know if the examples and new patch answer your question/solve the 
problem or, if not, what parts of the problem persist.


Jim


________________________________
From: Artem Pisarenko <address@hidden>
Sent: Wednesday, August 29, 2018 5:45 AM
To: address@hidden
Cc: Nutaro, James J.
Subject: problems achieving "fullspeed" system emulation (TCG, icount, sleep 
off, rtc is vm)

I'm trying to integrate linux/x86 system emulation into simulation model of 
large system using QEmu running with options "-icount 1,sleep=off -rtc 
clock=vm" (non-KVM mode, of course) modified with patch "qqq: module for 
synchronizing with a simulation" by James Nutaro. Target system expected to be 
isolated from host environment as it has to be deterministic, particulary, it 
runs independently of real time on the host. The behavior I'm expecting with 
given options is that QEmu will "drive" virtual clock as fast as it can 
relative to realtime clock.

I will use term 'speed' to describe virtual clock to realtime clock average 
ratio observed while running target. Speed expected to be limited only by host 
performance.

As far as I understand, very(!) simplified model of how QEmu should work may be 
described with following single-threaded pseudo-code:

while (1) {
  if ((virtualclock % icount) == 0)
    execute_vcpu_instruction();
  if ((virtualclock % VIRTUAL_TO_REAL_TIME_RATIO) == 0)
    increment_rtc_time_seen_by_guest();
  execute_all_other_hardware_activity(virtualclock);
  process_events(); // all other stuff, such as monitor commands, UI, etc..
  virtualclock++;
}

In other words, when it runs linux target system which ideally does nothing but 
servicing console commands (i.e. being idle), each time I type 'hwclock' 
command to console, I should see huge advancement of shown time relative to 
real world time being passed between typing, i.e. speed should strive for 
infinity (given that host benchmark 'score' strives for infinity and target OS 
strives for sleep cpu all time).

But actually I'm observing very and very strange behavior, described in 
following scenarios:
1. Both target and host are idle. Target time runs only ~5 times faster than 
real time. Host system monitor shows almost no cpu usage by QEmu process (so it 
explains such poor speed-up).
2. Qemu options contains "-nodefaults". If I'm not adding either "-display none 
-device qxl-vga" or "-display sdl", I''m getting target time much more slower, 
even slower than real time. Target system doesn't even access display hardware 
at all. Option combinations are exactly as I provided (i.e. replacing sdl with 
gtk gives different results). Qemu process still doesn't eat much cpu resources.
3. And finally the most shocking fact. When I'm loading host cpu in parallel 
(e.g. by running any benchmark/stress test on all cores), QEmu process activity 
increases to 2-4% and target time speeds up to ~100 times !!! Less cores loaded 
- less speed-up.

These results after several days of investigating makes my brain screaming: 
"WTF!?"

Something throttles virtual clock advancement, either intentionally or 
indirectly, or by mistake... Looks like pseudo-code above contains calls doing 
some synchronization with host system clock/timer/events/etc as a result of:
- some emulated hardware misbehavior;
- poor overall design of internal event/timers processing;
- some wrong optimization technique not accounting such usage scenario;
- some legacy piece of code doing tricky sabotage;
- ...

Or maybe my expectations are wrong and QEmu just isn't intended to be used in 
such way ?

QEmu: 2.12.0 (built with 'configure --target-list=x86_64-softmmu 
--enable-debug'), patched with https://patchwork.kernel.org/patch/10124685/
Target system hardware configuration as minimal as possible.
Host system: 64-bit, Ubuntu 16.04, Intel Core i7-7700 CPU.
--

? ?????????,
  ????? ?????????