Re: [PATCH v7 3/9] i386: hardcode supported eVMCS version to '1'

[Vitaly Kuznetsov]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Jun 04, 2021 at 09:28:15AM +0200, Vitaly Kuznetsov wrote:
>> Eduardo Habkost <ehabkost@redhat.com> writes:
>> 
>> > On Thu, Jun 03, 2021 at 01:48:29PM +0200, Vitaly Kuznetsov wrote:
>> >> Currently, the only eVMCS version, supported by KVM (and described in 
>> >> TLFS)
>> >> is '1'. When Enlightened VMCS feature is enabled, QEMU takes the supported
>> >> eVMCS version range (from KVM_CAP_HYPERV_ENLIGHTENED_VMCS enablement) and
>> >> puts it to guest visible CPUIDs. When (and if) eVMCS ver.2 appears a
>> >> problem on migration is expected: it doesn't seem to be possible to 
>> >> migrate
>> >> from a host supporting eVMCS ver.2 to a host, which only support eVMCS
>> >> ver.1.
>> >
>> > Isn't it possible and safe to expose eVMCS ver.1 to the guest on
>> > a host that supports ver.2?
>> 
>> We expose the supported range, guest is free to use any eVMCS version in
>> the range (see below):
>
> Oh, I didn't notice the returned value was a range.
>
>> 
>> >
>> >> 
>> >> Hardcode eVMCS ver.1 as the result of 'hv-evmcs' enablement for now. Newer
>> >> eVMCS versions will have to have their own enablement options (e.g.
>> >> 'hv-evmcs=2').
>> >> 
>> >> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> >> ---
>> >>  docs/hyperv.txt       |  2 +-
>> >>  target/i386/kvm/kvm.c | 16 +++++++++++-----
>> >>  2 files changed, 12 insertions(+), 6 deletions(-)
>> >> 
>> >> diff --git a/docs/hyperv.txt b/docs/hyperv.txt
>> >> index a51953daa833..000638a2fd38 100644
>> >> --- a/docs/hyperv.txt
>> >> +++ b/docs/hyperv.txt
>> >> @@ -170,7 +170,7 @@ Recommended: hv-frequencies
>> >>  3.16. hv-evmcs
>> >>  ===============
>> >>  The enlightenment is nested specific, it targets Hyper-V on KVM guests. 
>> >> When
>> >> -enabled, it provides Enlightened VMCS feature to the guest. The feature
>> >> +enabled, it provides Enlightened VMCS version 1 feature to the guest. 
>> >> The feature
>> >>  implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V)
>> >>  hypervisors making L2 exits to the hypervisor faster. The feature is 
>> >> Intel-only.
>> >>  Note: some virtualization features (e.g. Posted Interrupts) are disabled 
>> >> when
>> >> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
>> >> index c676ee8b38a7..d57eede5dc81 100644
>> >> --- a/target/i386/kvm/kvm.c
>> >> +++ b/target/i386/kvm/kvm.c
>> >> @@ -1490,13 +1490,19 @@ static int hyperv_init_vcpu(X86CPU *cpu)
>> >>          ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
>> >>                                    (uintptr_t)&evmcs_version);
>> >>  
>> >> -        if (ret < 0) {
>> >> -            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
>> >> -                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
>> >> +        /*
>> >> +         * KVM is required to support EVMCS ver.1. as that's what 
>> >> 'hv-evmcs'
>> >> +         * option sets. Note: we hardcode the maximum supported eVMCS 
>> >> version
>> >> +         * to '1' as well so 'hv-evmcs' feature is migratable even when 
>> >> (and if)
>> >> +         * ver.2 is implemented. A new option (e.g. 'hv-evmcs=2') will 
>> >> then have
>> >> +         * to be added.
>> >> +         */
>> >> +        if (ret < 0 || (uint8_t)evmcs_version > 1) {
>> >
>> > Wait, do you really want to get a fatal error every time, after a
>> > kernel upgrade?
>> >
>> 
>> Here, evmcs_version (returned by kvm_vcpu_enable_cap()) represents a
>> *range* of supported eVMCS versions:
>> 
>> (evmcs_highest_supported_version << 8) | evmcs_lowest_supported_version
>> 
>> Currently, this is 0x101 [1..1] range.
>> 
>> The '(uint8_t)evmcs_version > 1' check here means 'eVMCS v1' is no
>> longer supported by KVM. This is not going to happen any time soon, but
>> I can imagine in 10 years or so we'll be dropping v1 so the range (in
>> theory) can be [10..2] -- which would mean eVMCS ver. 1 is NOT
>> supported. And we can't proceed then.
>
> Where is this documented?  The only reference to
> KVM_CAP_HYPERV_ENLIGHTENED_VMCS I've found in linux/Documentation is this
> single sentence:
>
> | - HYPERV_CPUID_NESTED_FEATURES leaf and HV_X64_ENLIGHTENED_VMCS_RECOMMENDED
> |  feature bit are only exposed when Enlightened VMCS was previously enabled
> |  on the corresponding vCPU (KVM_CAP_HYPERV_ENLIGHTENED_VMCS).
>

It seems that KVM_CAP_HYPERV_ENLIGHTENED_VMCS's documentation is indeed
missing in KVM.

>
>> >>          ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
>> >>                                    (uintptr_t)&evmcs_version);
>> >>  
>> >> -        if (ret < 0) {
>> >> -            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
>> >> -                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
>> >> +        /*
>> >> +         * KVM is required to support EVMCS ver.1. as that's what 
>> >> 'hv-evmcs'
>> >> +         * option sets. Note: we hardcode the maximum supported eVMCS 
>> >> version
>> >> +         * to '1' as well so 'hv-evmcs' feature is migratable even when 
>> >> (and if)
>> >> +         * ver.2 is implemented. A new option (e.g. 'hv-evmcs=2') will 
>> >> then have
>> >> +         * to be added.
>> >> +         */
>> >> +        if (ret < 0 || (uint8_t)evmcs_version > 1) {
>
> Can we start with something that won't need to be rewritten after we change 
> the
> guest evmcs version range?  e.g.:
>
>     static bool evmcs_version_supported(uint16_t version, uint32_t supported)
>     {
>         uint8_t min_ver = version;
>         uint8_t max_ver = version >> 8;
>         uint8_t min_supported = supported;
>         uint8_t max_supported = supported >> 8;
>         return (min_ver >= min_supported) && (max_ver <= max_supported);
>     }
>     ...
>     #define DEFAULT_EVMCS_VERSION ((1 << 8) | 1)
>     ...
>     uint16_t evmcs_version = DEFAULT_EVMCS_VERSION;
>     int ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
>                               (uintptr_t)&supported_evmcs_version);
>     if (ret < 0) {
>         fprintf(...)
>         return ret;
>     }
>     if (!evmcs_version_supported(evmcs_version, supported_evmcs_version)) {
>         fprintf(...)
>         return -ENOTSUP;
>     }
>     cpu->hyperv_nested[0] = evmcs_version;
>

Sure, I don't have anything against an attempt to make this future
proof. Will use this in v8, thanks!

-- 
Vitaly