Re: [PATCH] ppc/spapr: Add support for implement support for H_SCM_HEALTH

[Vaibhav Jain]

Thanks for looking into this patch Greg. My responses below inline.


Greg Kurz <groug@kaod.org> writes:

> Hi Vaibhav,
>
> Great to see you around :-)

:-)

>
> On Mon, 29 Mar 2021 21:52:59 +0530
> Vaibhav Jain <vaibhav@linux.ibm.com> wrote:
>
>> Add support for H_SCM_HEALTH hcall described at [1] for spapr
>> nvdimms. This enables guest to detect the 'unarmed' status of a
>> specific spapr nvdimm identified by its DRC and if its unarmed, mark
>> the region backed by the nvdimm as read-only.
>> 
>
> Any chance that you can provide the documentation of this new hcall ?
>
H_SCM_HEALTH specifications is already documented in linux kernel
documentation at
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/powerpc/papr_hcalls.rst

That documentation was added when kernel support for H_SCM_HEALTH hcall
support was implemented in 5.9 kernel. 

>> The patch adds h_scm_health() to handle the H_SCM_HEALTH hcall which
>> returns two 64-bit bitmaps (health bitmap, health bitmap mask) derived
>> from 'struct nvdimm->unarmed' member.
>> 
>> Linux kernel side changes to enable handling of 'unarmed' nvdimms for
>> ppc64 are proposed at [2].
>> 
>> References:
>> [1] "Hypercall Op-codes (hcalls)"
>>     
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/powerpc/papr_hcalls.rst
>> 
>> [2] "powerpc/papr_scm: Mark nvdimm as unarmed if needed during probe"
>>     
>> https://lore.kernel.org/linux-nvdimm/20210329113103.476760-1-vaibhav@linux.ibm.com/
>> 
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
>> ---
>>  hw/ppc/spapr_nvdimm.c  | 30 ++++++++++++++++++++++++++++++
>>  include/hw/ppc/spapr.h |  4 ++--
>>  2 files changed, 32 insertions(+), 2 deletions(-)
>> 
>> diff --git a/hw/ppc/spapr_nvdimm.c b/hw/ppc/spapr_nvdimm.c
>> index b46c36917c..e38740036d 100644
>> --- a/hw/ppc/spapr_nvdimm.c
>> +++ b/hw/ppc/spapr_nvdimm.c
>> @@ -31,6 +31,13 @@
>>  #include "qemu/range.h"
>>  #include "hw/ppc/spapr_numa.h"
>>  
>> +/* DIMM health bitmap bitmap indicators */
>> +/* SCM device is unable to persist memory contents */
>> +#define PAPR_PMEM_UNARMED (1ULL << (63 - 0))
>
> This looks like PPC_BIT(0).
>
Yes, right. Will update the patch in v2 to use the PPC_BIT macro.

>> +
>> +/* Bits status indicators for health bitmap indicating unarmed dimm */
>> +#define PAPR_PMEM_UNARMED_MASK (PAPR_PMEM_UNARMED)
>> +
>>  bool spapr_nvdimm_validate(HotplugHandler *hotplug_dev, NVDIMMDevice 
>> *nvdimm,
>>                             uint64_t size, Error **errp)
>>  {
>> @@ -467,6 +474,28 @@ static target_ulong h_scm_unbind_all(PowerPCCPU *cpu, 
>> SpaprMachineState *spapr,
>>      return H_SUCCESS;
>>  }
>>  
>> +static target_ulong h_scm_health(PowerPCCPU *cpu, SpaprMachineState *spapr,
>> +                                 target_ulong opcode, target_ulong *args)
>> +{
>> +    uint32_t drc_index = args[0];
>> +    SpaprDrc *drc = spapr_drc_by_index(drc_index);
>> +    NVDIMMDevice *nvdimm;
>> +
>> +    if (drc && spapr_drc_type(drc) != SPAPR_DR_CONNECTOR_TYPE_PMEM) {
>> +        return H_PARAMETER;
>> +    }
>> +
>> +    nvdimm = NVDIMM(drc->dev);
>
> Yeah as already suggested by Shiva, drc->dev should be checked like
> in h_scm_bind_mem().
>
Yes, will send a v2 with this case handled.

>> +
>> +    /* Check if the nvdimm is unarmed and send its status via health 
>> bitmaps */
>> +    args[0] = nvdimm->unarmed ? PAPR_PMEM_UNARMED_MASK : 0;
>> +
>
> Shouldn't ^^ use PAPR_PMEM_UNARMED then ?
>
>> +    /* health bitmap mask same as the health bitmap */
>> +    args[1] = args[0];
>> +
>
> If so, it seems that PAPR_PMEM_UNARMED_MASK isn't even needed.

Definition of these defines are similar to what kernel implementation
uses at
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/platforms/pseries/papr_scm.c#n53

Since unarmed condition can also arise due to an unhealthy nvdimm hence
the kernel implementation uses a mask thats composed of two bits
PPC_BIT(0) and PPC_BIT(6) being set. Though we arent using PPC_BIT(6)
right now in qemu, it will change in future when better nvdimm health
reporting will be done. Hence kept the PPC_BIT(0) define as well as the
mask to mimic the kernel definitions.

>
> Having access to the excerpts from the PAPR addendum that describes
> this hcall would _really_ help in reviewing.
>
The kernel documentation for H_SCM_HEALTH mentioned above captures most
if not all parts of the PAPR addendum for this hcall. I believe it
contains enough information to review the patch. If you still need more
info than please let me know.


>> +    return H_SUCCESS;
>> +}
>> +
>>  static void spapr_scm_register_types(void)
>>  {
>>      /* qemu/scm specific hcalls */
>> @@ -475,6 +504,7 @@ static void spapr_scm_register_types(void)
>>      spapr_register_hypercall(H_SCM_BIND_MEM, h_scm_bind_mem);
>>      spapr_register_hypercall(H_SCM_UNBIND_MEM, h_scm_unbind_mem);
>>      spapr_register_hypercall(H_SCM_UNBIND_ALL, h_scm_unbind_all);
>> +    spapr_register_hypercall(H_SCM_HEALTH, h_scm_health);
>>  }
>>  
>>  type_init(spapr_scm_register_types)
>> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
>> index 47cebaf3ac..18859b9ab2 100644
>> --- a/include/hw/ppc/spapr.h
>> +++ b/include/hw/ppc/spapr.h
>> @@ -538,8 +538,8 @@ struct SpaprMachineState {
>>  #define H_SCM_BIND_MEM          0x3EC
>>  #define H_SCM_UNBIND_MEM        0x3F0
>>  #define H_SCM_UNBIND_ALL        0x3FC
>> -
>> -#define MAX_HCALL_OPCODE        H_SCM_UNBIND_ALL
>> +#define H_SCM_HEALTH            0x400
>> +#define MAX_HCALL_OPCODE        H_SCM_HEALTH
>>  
>>  /* The hcalls above are standardized in PAPR and implemented by pHyp
>>   * as well.
>

-- 
Cheers
~ Vaibhav