Re: [Qemu-devel] Gives user ability to select endian format for video display - fixes Mac OS X guest color issue.

[Programmingkid]

On Jan 6, 2015, at 12:30 PM, Peter Maydell wrote:

On 6 January 2015 at 17:19, Programmingkid <address@hidden> wrote:

After investigating the TARGET_WORDS_BIGENDIAN code, I noticed

that s->default_endian_fb was being set to true. So I undefined

the macro and then ran QEMU. The i386 target showed no change

in colors. The ppc target still had the same incorrect colors.

Sorry, this doesn't fix the problem.

What macro? You don't want to undefine TARGET_WORDS_BIGENDIAN,
that will wreak all kinds of havoc. Just test with manually
setting default_endian_fb to false.

-- PMM

Just tried that. It didn't fix the color problem. Here are the test I have done so far:

Experiment: Disable preprocessor code so that thebool byteswap = !s->big_endian_fb code is used.  Line 1440. 

Result: No change in colors for Mac OS X guest. They are still incorrect. 

Experiment: Set the HOST_WORDS_BIGENDIAN macro at line 94. 

Result: Did not change the colors. They are still messed up. It made all the colors in the PC emulator look weird - like a Nintendo cartridge not fully seated correctly. But it then changed back to the normal color layout. 

Experiment: undefine TARGET_WORDS_BIGENDIAN at line 2155 in vga.c. 

Result: PC emulator shows no change in colors - good. Mac emulator shows the same incorrect colors -bad. 

Experiment: remove the ! from byteswap in vga.c:1495. 

Result: The colors are still incorrect in qemu-system-ppc with a Mac OS X guest. Colors are unaffected in the i386 target.

Experiment: Just manually set default_endian_fb to false. 

Result: Did not fix the problem. Colors still show up incorrectly in the ppc emulator. 

Sorry but I don't think this is a bug with QEMU. I think the pixel format is suppose to be in the little endian format. IBM developed this standard for this PCs which used little endian x86 processors. Why would they want to spend the time and money on making their vga cards work on non-pc/2 compatible computers? They simply had no reason to support the big endian format.