[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug 1856335] Re: Cache Layout wrong on many Zen Arch CPUs
From: |
Jan Klos |
Subject: |
[Bug 1856335] Re: Cache Layout wrong on many Zen Arch CPUs |
Date: |
Fri, 12 Jun 2020 08:53:16 -0000 |
The problem is caused by the fact that with Ryzen CPUs with disabled
cores, the APIC IDs are not sequential on host - in order for cache
topology to be configured properly, there is a 'hole' in APIC ID and
core ID numbering (I have added full output of cpuid for my 3900X).
Unfortunately, adding holes to the numbering is the only way to achieve
what is needed for 3 cores per CCX as CPUID Fn8000_001D_EAX
NumSharingCache parameter rounds to powers of two (for Ryzen 3100 with
2 cores per CCX, lowering NumSharingCache should also work, correctly
setting the L3 cache cores with their IDs still being sequential).
A small hack in x86_apicid_from_topo_ids() in include/hw/i386/topology.h
can introduce a correct numbering (at least if you do not have epyc set
as your cpu, then _epyc variant of the functions are used). But to fix
this properly will probably require some thought - maybe introduce the
ability to assign APIC IDs directly somehow? Or the ability to specify
the 'holes' somehow in the -smt param, or maybe -cpu host,topoext=on
should do this automatically? I don't know...
e.g. For 3 core per CCX CPUs, to fix this, at
include/hw/i386/topology.h:220 change:
(topo_ids->core_id << apicid_core_offset(topo_info)) |
to
((topo_ids->core_id + (topo_ids->core_id / 3)) <<
apicid_core_offset(topo_info)) |
The cache topology is now correct (-cpu
host,topoext=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,host-cache-info=on
-smp 18,sockets=1,dies=1,cores=9,threads=2), even in Windows:
Logical Processor to Cache Map:
**---------------- Data Cache 0, Level 1, 32 KB, Assoc 8,
LineSize 64
**---------------- Instruction Cache 0, Level 1, 32 KB, Assoc 8,
LineSize 64
**---------------- Unified Cache 0, Level 2, 512 KB, Assoc 8,
LineSize 64
******------------ Unified Cache 1, Level 3, 16 MB, Assoc 16,
LineSize 64
--**-------------- Data Cache 1, Level 1, 32 KB, Assoc 8,
LineSize 64
--**-------------- Instruction Cache 1, Level 1, 32 KB, Assoc 8,
LineSize 64
--**-------------- Unified Cache 2, Level 2, 512 KB, Assoc 8,
LineSize 64
----**------------ Data Cache 2, Level 1, 32 KB, Assoc 8,
LineSize 64
----**------------ Instruction Cache 2, Level 1, 32 KB, Assoc 8,
LineSize 64
----**------------ Unified Cache 3, Level 2, 512 KB, Assoc 8,
LineSize 64
------**---------- Data Cache 3, Level 1, 32 KB, Assoc 8,
LineSize 64
------**---------- Instruction Cache 3, Level 1, 32 KB, Assoc 8,
LineSize 64
------**---------- Unified Cache 4, Level 2, 512 KB, Assoc 8,
LineSize 64
------******------ Unified Cache 5, Level 3, 16 MB, Assoc 16,
LineSize 64
--------**-------- Data Cache 4, Level 1, 32 KB, Assoc 8,
LineSize 64
--------**-------- Instruction Cache 4, Level 1, 32 KB, Assoc 8,
LineSize 64
--------**-------- Unified Cache 6, Level 2, 512 KB, Assoc 8,
LineSize 64
----------**------ Data Cache 5, Level 1, 32 KB, Assoc 8,
LineSize 64
----------**------ Instruction Cache 5, Level 1, 32 KB, Assoc 8,
LineSize 64
----------**------ Unified Cache 7, Level 2, 512 KB, Assoc 8,
LineSize 64
------------**---- Data Cache 6, Level 1, 32 KB, Assoc 8,
LineSize 64
------------**---- Instruction Cache 6, Level 1, 32 KB, Assoc 8,
LineSize 64
------------**---- Unified Cache 8, Level 2, 512 KB, Assoc 8,
LineSize 64
------------****** Unified Cache 9, Level 3, 16 MB, Assoc 16,
LineSize 64
** Attachment added: "cpuid.txt"
https://bugs.launchpad.net/qemu/+bug/1856335/+attachment/5383184/+files/cpuid.txt
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1856335
Title:
Cache Layout wrong on many Zen Arch CPUs
Status in QEMU:
New
Bug description:
AMD CPUs have L3 cache per 2, 3 or 4 cores. Currently, TOPOEXT seems
to always map Cache ass if it was an 4-Core per CCX CPU, which is
incorrect, and costs upwards 30% performance (more realistically 10%)
in L3 Cache Layout aware applications.
Example on a 4-CCX CPU (1950X /w 8 Cores and no SMT):
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>EPYC-IBPB</model>
<vendor>AMD</vendor>
<topology sockets='1' cores='8' threads='1'/>
In windows, coreinfo reports correctly:
****---- Unified Cache 1, Level 3, 8 MB, Assoc 16, LineSize 64
----**** Unified Cache 6, Level 3, 8 MB, Assoc 16, LineSize 64
On a 3-CCX CPU (3960X /w 6 cores and no SMT):
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>EPYC-IBPB</model>
<vendor>AMD</vendor>
<topology sockets='1' cores='6' threads='1'/>
in windows, coreinfo reports incorrectly:
****-- Unified Cache 1, Level 3, 8 MB, Assoc 16, LineSize 64
----** Unified Cache 6, Level 3, 8 MB, Assoc 16, LineSize 64
Validated against 3.0, 3.1, 4.1 and 4.2 versions of qemu-kvm.
With newer Qemu there is a fix (that does behave correctly) in using the dies
parameter:
<qemu:arg value='cores=3,threads=1,dies=2,sockets=1'/>
The problem is that the dies are exposed differently than how AMD does
it natively, they are exposed to Windows as sockets, which means, that
if you are nto a business user, you can't ever have a machine with
more than two CCX (6 cores) as consumer versions of Windows only
supports two sockets. (Should this be reported as a separate bug?)
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1856335/+subscriptions
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Bug 1856335] Re: Cache Layout wrong on many Zen Arch CPUs,
Jan Klos <=