Re: [Qemu-ppc] [PATCH 10/25] spapr: add MMIO handlers for the XIVE interrupt sources

[Cédric Le Goater]

On 11/30/2017 04:28 AM, David Gibson wrote:
> On Wed, Nov 29, 2017 at 05:23:25PM +0100, Cédric Le Goater wrote:
>> On 11/29/2017 02:56 PM, Cédric Le Goater wrote:
>>>>>>> +    switch (offset) {
>>>>>>> +    case 0:
>>>>>>> +        spapr_xive_source_eoi(xive, lisn);
>>>>>>
>>>>>> Hrm.  I don't love that you're dealing with clearing that LSI bit
>>>>>> here, but setting it at a different level.
>>>>>>
>>>>>> The state machines are doing my head in a bit, is there any way
>>>>>> you could derive the STATUS_SENT bit from the PQ bits?
>>>>>
>>>>> Yes. I should. 
>>>>>
>>>>> I am also lacking a guest driver to exercise these LSIs so I didn't
>>>>> pay a lot of attention to level interrupts. Any idea ?
>>>>
>>>> How about an old-school emulated PCI device?  Maybe rtl8139?
>>>
>>> Perfect. The current model is working but I will see how I can 
>>> improve it to use the PQ bits instead.
>>
>> Using the PQ bits is simplifying the model but we still have to 
>> maintain an array to store the IRQ type. 
>>
>> There are 3 unused bits in the IVE descriptor, bits[1-3]:  
>>
>>   #define IVE_VALID       PPC_BIT(0)
>>   #define IVE_EQ_BLOCK    PPC_BITMASK(4, 7)        /* Destination EQ block# 
>> */
>>   #define IVE_EQ_INDEX    PPC_BITMASK(8, 31)       /* Destination EQ index */
>>   #define IVE_MASKED      PPC_BIT(32)              /* Masked */
>>   #define IVE_EQ_DATA     PPC_BITMASK(33, 63)      /* Data written to the EQ 
>> */
>>
>> We could hijack one of them to store the LSI type and get rid of 
>> the type array. Would you object to that ?
> 
> Hrm.  These IVE bits are architected, aren't they?  

Yes and unused.

> In which case I'd
> be wary of stealing a reserved bit in case of future extensions.
> 
> I'm wondering if we want another word / structure for storing
> non-architected, implementation specific flags or info.

That's what is done with the status array. As migration will put 
pressure on future changes, may be, we should go for an extra 
word for these model non-architected needs.
 
> How does this work at the hardware level?  Presumbly the actual
> hardware components don't communicate with the XIVE to request edge or
> level.  

No.

> So how does it know?  Specific ranges for LSIs? 
 
When OPAL allocates interrupt numbers for a device, it records 
their type to handle the EOI a little differently for LSIs. 
For sPAPR, it is really the same, the hcall H_INT_GET_SOURCE_INFO 
gives the interrupt type to the guest and the EOI is handled 
with a specific load. A part from that, the LSI interrupts follow 
the same path as the MSI.
 
The device controller must have some extra logic to handle the 
level for these interrupts but I am no expert in the domain. 

> If that we should probably do the same.

Modeling the LSI level with the PQ bits looks fine. We just need 
to store the IRQ type information somewhere under the XIVE object. 
We can keep it in a byte array or a bitmap to reduce the size. 
But if we foresee additional state to store, we might want to use 
the byte array directly.

Thanks,

C.