Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Andre Przywara]

Anthony Liguori wrote:
> Gleb Natapov wrote:
> > > I thought KVM emulates the syscall instruction?  I swear I've seen
> > > patches for that.
> > >
> > >     
> > It is. Starting from 2.6.32.
> >   
>
> Okay, so this is a performance vs. migration compatibility issue then?
>
> BTW, couldn't we just not advertise syscall in cpuid?
Looks like a good idea, but at least Linux does not care ;-)
In 64-on-64 bit mode syscall is unconditionally used, which is OK, since 
it is architectural to AMD64. Intel integrated this, too.
In 32bit legacy mode AMD supports both syscall and sysenter, but mostly 
sysenter is used when the SEP CPUID bit is on. Intel does not support 
syscall, but this also is not advertised when CPUID is executed in 32bit.
In 32-on-64 compat mode the support is exclusive: AMD does not support 
sysenter, Intel does not support syscall. Since the AMD CPUID 
advertising is (at least) a bit "unfortunate" here ;-), Linux does not 
care about the bits, but unconditionally uses syscall on AMD and 
sysenter on Intel. See arch/x86/kernel/cpu/intel.c:early_init_intel():
#ifdef CONFIG_X86_64
    set_cpu_cap(c, X86_FEATURE_SYSENTER32);
#else
A similiar line is in cpu/amd.c:early_init_amd().
Then in arch/x86/vdso/vdso32-setup.c:sysenter_setup() these bit are used 
to populate the VDSO page. In the whole process CPUID feature bits are 
not honored, so you cannot get around it.
The only solution would be to inject neither AMD or Intel (I used 
VirtualLinux) in the past. This works fine for Linux (even better than 
the other two, since no vendor-specific workarounds are triggered), but 
64bit Windows refuses to boot on anything other than AuthenticAMD, 
GenuineIntel or CentaurHauls. Linux would then use int80h, which is 
slower than syscall or sysenter, but still much faster than emulation.

So given the whole mess I'd recommend to copy over Avi's "propagate 
local vendor" solution, as this solves the problem for 99.9+% of the 
cases. The remaining cases are cross-vendor migration, where we are fine 
with the 2.6.32 syscall/sysenter emulation. Although this is not as slow 
as one thinks (I measured less than 5% slowdown in kernbench), I'd not 
recommend it for common usage except cross-vendor migration.

BTW: Windows is not affected at all with this compat mode issue, because 
32bit applications switch to 64bit mode before doing the actual system 
call (which uses the 64bit syscall instruction).

> That should fix 
> it too without sacrificing migration compatibility.  We get a slight 
> slowdown on AMD hosts
I think there would be no slowdown. At least Linux only cares about this 
bit if in legacy 32bit mode, but here it uses sysenter anyway if this is 
present:
From arch/x86/vdso/vdso32-setup.c:
#else  /* CONFIG_X86_32 */
#define vdso32_sysenter()   (boot_cpu_has(X86_FEATURE_SEP))
#define vdso32_syscall()    (0)

> but that's probably minor compared to the cost of 
> using emulated syscall on Intel hosts.


Regards,
Andre.


--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632