Re: [PATCH v3] spapr: Add a new level of NUMA for GPUs

[Reza Arbab]

On Mon, May 25, 2020 at 03:05:50PM +1000, David Gibson wrote:
> On Fri, May 22, 2020 at 02:53:33PM -0500, Reza Arbab wrote:
> > --- a/hw/ppc/spapr_pci_nvlink2.c
> > +++ b/hw/ppc/spapr_pci_nvlink2.c
> > @@ -362,7 +362,7 @@ void spapr_phb_nvgpu_ram_populate_dt(SpaprPhbState *sphb, 
> > void *fdt)
> >          uint32_t associativity[] = {
> >              cpu_to_be32(0x4),
> >              SPAPR_GPU_NUMA_ID,
> > -            SPAPR_GPU_NUMA_ID,
> > +            cpu_to_be32(nvslot->numa_id),
> >              SPAPR_GPU_NUMA_ID,
> >              cpu_to_be32(nvslot->numa_id)
>
>
> This doesn't look quite right.  In the new case we'll get {
> GPU_NUMA_ID, nvslot->numa_id, GPU_NUMA_ID, nvslot->numa_id }.

The associativity reference points are 4 (and now 2), so this is what we 
want. I think what you've noticed is that reference points are 1-based 
ordinals:

        "...the “ibm,associativity-reference-points” property indicates
boundaries between associativity domains presented by the 
“ibm,associativity” property containing “near” and “far” resources. The 
first such boundary in the list represents the 1 based ordinal in the 
associativity lists of the most significant boundary, with subsequent 
entries indicating progressively less significant boundaries."

--
Reza Arbab