[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [RFC PATCH 1/1] i386: Remove features from Epyc-Milan cpu
|
From: |
Dr. David Alan Gilbert |
|
Subject: |
Re: [RFC PATCH 1/1] i386: Remove features from Epyc-Milan cpu |
|
Date: |
Tue, 1 Feb 2022 09:20:41 +0000 |
|
User-agent: |
Mutt/2.1.5 (2021-12-30) |
* Babu Moger (babu.moger@amd.com) wrote:
>
> On 1/31/22 14:18, Leonardo Bras Soares Passos wrote:
> > On Mon, Jan 31, 2022 at 3:04 PM Daniel P. Berrangé <berrange@redhat.com>
> > wrote:
> >> On Mon, Jan 31, 2022 at 02:56:38PM -0300, Leonardo Bras Soares Passos
> >> wrote:
> >>> Hello Daniel,
> >>>
> >>> On Mon, Jan 31, 2022 at 6:08 AM Daniel P. Berrangé <berrange@redhat.com>
> >>> wrote:
> >>>> CC'ing Babu Moger who aded the Milan CPU model.
> >>>>
> >>>> On Sat, Jan 29, 2022 at 07:23:37AM -0300, Leonardo Bras wrote:
> >>>>> While trying to bring a VM with EPYC-Milan cpu on a host with
> >>>>> EPYC-Milan cpu (EPYC 7313), the following warning can be seen:
> >>>>>
> >>>>> qemu-system-x86_64: warning: host doesn't support requested feature:
> >>>>> CPUID.07H:EBX.erms [bit 9]
> >>>>> qemu-system-x86_64: warning: host doesn't support requested feature:
> >>>>> CPUID.07H:EDX.fsrm [bit 4]
> >>>>>
> >>>>> Even with this warning, the host goes up.
> >>>>>
> >>>>> Then, grep'ing cpuid output on both guest and host, outputs:
> >>>>>
> >>>>> extended feature flags (7):
> >>>>> enhanced REP MOVSB/STOSB = false
> >>>>> fast short REP MOV = false
> >>>>> (simple synth) = AMD EPYC (3rd Gen) (Milan B1) [Zen 3], 7nm
> >>>>> brand = "AMD EPYC 7313 16-Core Processor "
> >>>>>
> >>>>> This means that for the same -cpu model (EPYC-Milan), the vcpu may or
> >>>>> may
> >>>>> not have the above feature bits set, which is usually not a good idea
> >>>>> for
> >>>>> live migration:
> >>>>> Migrating from a host with these features to a host without them can
> >>>>> be troublesome for the guest.
> >>>>>
> >>>>> Remove the "optional" features (erms, fsrm) from Epyc-Milan, in order to
> >>>>> avoid possible after-migration guest issues.
> >>>> Babu, can you give some insight into availability of erms / fsrm
> >>>> features across the EPYC 3rd gen CPU line. Is this example missing
> >>>> erms/fsrm an exception, or common place ?
>
> AMD supports fsrm and erms in EPYC 3rd gen CPUs. But there is some
> inconsistency in enabling these features in the BIOS. Some BIOSes enable
> it automatically and some BIOSes don't. But there a BIOS option
> (in ADVANCED -> AMD CBS) to enable/disable manually. We are working
> internally to find out the going forward strategy for these features. We
> will update the code when we find out about it.
>
> We know it is causing little bit of annoyance to the users. But as far as
> we know it should not cause migration issues as already discussed.
Given what Leo and Dan have described it'll probably cause some odd CPU
model usage for us, and possibly failures for customers trying to
migrate between hosts in a farm where the BIOS is configured differently
on the two hosts.
Having said that, these flags are interesting.
If I understand correctly they're just hints to the guest whether to
use rep movsb or not - there's no extra state, no change in semantics -
so in theory if they mismatch it doesn't matter.
Dave
> thanks
>
>
> >>>>
> >>>>> Signed-off-by: Leonardo Bras <leobras@redhat.com>
> >>>>> ---
> >>>>>
> >>>>> Does this make sense? Or maybe I am missing something here.
> >>>>>
> >>>>> Having a kvm guest running with a feature bit, while the host
> >>>>> does not support it seems to cause a possible break the guest.
> >>>> The guest won't see the feature bit - that warning message from QEMU
> >>>> is telling you that it did't honour the request to expose
> >>>> erms / fsrm - it has dropped them from the CPUO exposed to the guest.
> >>> Exactly.
> >>> What I meant here is:
> >>> 1 - Host with these feature bits start a VM with EPYC-Milan cpu (and
> >>> thus have those bits enabled)
> >>> 2 - Guest is migrated to a host such as the above, which does not
> >>> support those features (bits disabled), but does support EPYC-Milan
> >>> cpus (without those features).
> >>> 3 - The migration should be allowed, given the same cpu types. Then
> >>> either we have:
> >>> 3a : The guest vcpu stays with the flag enabled (case I tried to
> >>> explain above), possibly crashing if the new feature is used, or
> >>> 3b: The guest vcpu disables the flag due to incompatibility, which
> >>> may make the guest confuse due to cpu change, and even end up trying
> >>> to use the new feature on the guest, even if it's disabled.
> >> Neither should happen with a correctly written mgmt app in charge.
> >>
> >> When launching a QEMU process for an incoming migration, it is expected
> >> that the mgmt app has first queried QEMU on the source to get the precise
> >> CPU model + flags that were added/removed on the source. The QEMU on
> >> the target is then launched with this exact set of flags, and the
> >> 'check' flag is also set for -cpu. That will cause QEMU on the target
> >> to refuse to start unless it can give the guest the 100% identical
> >> CPUID to what has been requested on the CLI, and thus matching the
> >> source.
> >>
> >> Libvirt will ensure all this is done correctly. If not using libvirt
> >> then you've got a bunch of work to do to achieve this. It certainly
> >> isn't sufficient to merely use the same plain '-cpu' arg that the
> >> soruce was original booted with, unless you have 100% identical
> >> hardware, microcode, and software on both hosts, or the target host
> >> offers a superset of features.
> > Oh, that is very interesting! Thanks for sharing!
> >
> > Well, then at least one unexpected scenario should happen:
> > - VM with EPYC-Milan cpu, created in source host
> > - Source host with EPYC-Milan cpu. Support for 'extra features'
> > enabled ( erms / fsrm in this ex.)
> > - Target host with EPYC-Milan cpu. No support for 'extra features'.
> > Since the VM will be created with support for 'extra features', trying
> > to migrate from source host to target host should fail, right?
> >
> > Which is, IMHO, odd. I imagine questions like:
> > - "How does a host with EPYC-Milan cpu does not offer support to
> > receive a live migration of some VMs with EPYC-Milan cpu?", or even
> > - "If I can create a VM with EPYC-Milan cpu on that host, why can't I
> > receive (via migration) some VMs with EPYC-Milan CPU ?"
> >
> > But I am new to live migration, so maybe I am getting something wrong
> > regarding the cpu-model idea.
> >
> > Best regards,
> > Leo
> >
> >
> >
> >>
> >> Regards,
> >> Daniel
> >> --
> >> |:
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fberrange.com%2F&data=04%7C01%7Cbabu.moger%40amd.com%7C4b27f5f24f91458f4c1408d9e4f6d80a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637792571249191260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=t9LpUFrscExWZXhVkFWYLAuFDn%2FxEdmyFPEFAeSwwn8%3D&reserved=0
> >> -o-
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.flickr.com%2Fphotos%2Fdberrange&data=04%7C01%7Cbabu.moger%40amd.com%7C4b27f5f24f91458f4c1408d9e4f6d80a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637792571249191260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=TOhfcLvsf%2BOinXAsEH2pqL%2BUdhR6izv3Y1t5dv6n5Tw%3D&reserved=0
> >> :|
> >> |:
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt.org%2F&data=04%7C01%7Cbabu.moger%40amd.com%7C4b27f5f24f91458f4c1408d9e4f6d80a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637792571249191260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=V57dTTtEccVO7eUYrjB1CvHrY543bKMyt4R8Z5psgik%3D&reserved=0
> >> -o-
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffstop138.berrange.com%2F&data=04%7C01%7Cbabu.moger%40amd.com%7C4b27f5f24f91458f4c1408d9e4f6d80a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637792571249191260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GAkAOUbc0e5yDC92pUkZ50IywidWbZQHaZGUfgO28nI%3D&reserved=0
> >> :|
> >> |:
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fentangle-photo.org%2F&data=04%7C01%7Cbabu.moger%40amd.com%7C4b27f5f24f91458f4c1408d9e4f6d80a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637792571249191260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=x%2FCSGj5ru80b%2B6Gut17epf%2BGh9skGUdiQR6CUuxSdRA%3D&reserved=0
> >> -o-
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.instagram.com%2Fdberrange&data=04%7C01%7Cbabu.moger%40amd.com%7C4b27f5f24f91458f4c1408d9e4f6d80a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637792571249191260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=MOmJdAwgSNSz9%2B%2FKo4RD00k0GFdFmldUEeCrKHBFv2c%3D&reserved=0
> >> :|
> >>
> --
> Thanks
> Babu Moger
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK