Re: [Qemu-devel] Re: [PATCH 1/6] Make config space accessor host bus trapable

[Alexander Graf]

On 04.01.2010, at 23:59, Michael S. Tsirkin wrote:

> On Tue, Jan 05, 2010 at 09:51:48AM +1100, Benjamin Herrenschmidt wrote:
>> On Tue, 2010-01-05 at 00:25 +0200, Michael S. Tsirkin wrote:
>>> On Tue, Jan 05, 2010 at 08:53:52AM +1100, Benjamin Herrenschmidt wrote:
>>>> 
>>>>> Yes, but I think how you program your host to pci bridge is platform 
>>>>> specific,
>>>>> the standard (mostly) applies to what happens below the bridge.  There's
>>>>> no real standard for how PCI host bridge is connected to processor
>>>>> AFAIK, it's by luck we can share code there at all.
>>>> 
>>>> Well, yes and no ... there's a standard on how a PCI host bridge is
>>>> connected in the sense that how normal MMIO accesses go through in term
>>>> of endianness is well defined.
>>>> 
>>> 
>>> Go through where? From processor to PCI?
>>> Which spec do you refer to?
>> 
>> The PCI spec from memory :-) Byte swizzling for MMIO between a processor
>> and PCI bus is well defined.
> 
> Couldn't find it in spec.
> 
>> Now of course, since issuing config space cycles tend to be host-bridge
>> chip specific, everything is possible there :-) In -most- cases though,
>> they use a mechanism similar to x86 using the cf8/cfc ports or
>> equivalent mapped wherever the PIO region is mapped in MMIO space for
>> non-x86 processors, and thus end up with the exact same byte swizzling.
>> 
>> In fact, this is true of accesses to PCI devices as well. Take for
>> example, a device that has a 32-bit MMIO register. This register is
>> meant to appear as little endian (well, unless the device itself plays
>> tricks but most don't) whatever the host processor is. Thus an x86 host
>> will need no byteswap but a powerpc host (assuming the ppc is running in
>> BE mode) will.
>> 
>> This is why for example the base readl and writel function in Linux do
>> byteswap on powerpc.
>> 
>> This is important to understand that this is a property of how the PCI
>> bridge is connected to the host processor, such that the PCI "native"
>> byte order is preserved along with address invariance for sub-32-bit
>> quantities.
>> 
>> The endianness of the host bridge config space access register is thus
>> most of the time just a natural side effect of said register being part
>> of the bridge PIO space and thus getting the natural byteswap explained
>> above for a 32-bit LE register on PCI.
>> 
>> Cheers,
>> Ben.
> 
> So, it appears that this is not the case for many platforms: bridge
> itself does a byteswap to make devices behind it work according to spec,
> but this does not apply to programming bridge itself.
> 
> This seems common on BE platforms, this is why qemu has
> ifdef TARGET_WORDS_BIGENDIAN there IIUC.

IIRC qemu's mmio functions just pass the register value the guest had at that 
moment to the mmio function.

While that is pretty simple, it means that we behave different from real 
hardware. Real hardware has 2 components:

1) CPU
2) Device

The CPU sends an MMIO request in local byte order to the device. The device 
also has a native endianness - either BE or LE.
So what the byte swap here does is simply converting from wrong LE (what Linux 
thought the device needs) to a proper variable.

I'm not sure what the correct solution to this is. I'm inclined to suggest that 
mmio callbacks should get called with tswap'ed values. But then again the code 
as is works in quite a lot of cases and I don't want to spend months getting it 
back to where it is just because of a cosmetic change.

Alex