Missing IRQ with bmdma on ppc/mips/sparc? (was: Re: Emulating Solaris 10 on SPARC64 sun4u)

[BALATON Zoltan]

I've changed title to avoid derailing the original thread as this is more 
about pegasos2 issue now but left cc list for now. Let me know if you 
don't want to be cc'd.

On Mon, 10 Feb 2020, John Snow wrote:
> On 2/10/20 10:38 AM, BALATON Zoltan wrote:
> > On Sat, 8 Feb 2020, BALATON Zoltan wrote:
> > > Not sure if my problem I see on other machine emulation I'm working on
> > > is related at all but there's a possibility it might be. I got this
> > > with different arch (ppc but could also reproduce something similar
> > > with mips) and ide controller emulation (via-ide) but the PCI bmdma
> > > code is shared by CMD646, via-ide and sii3112 and also the ide-cdrom
> > > emulation is the same so if there's a bug in these that could cause
> > > similar problems for different components. Or it could be that we get
> > > similar symptoms due to different reasons in which case sorry for the
> > > distracion but maybe we can learn from the experience of each other
> > > even in that case.
> > >
> > > What I get is tracked here:
> > >
> > > https://osdn.net/projects/qmiga/ticket/38949

I've updated this ticket now with more detailed AmigaOS4 logs that also 
contains PIO regs except ide_data* to keep it managable. I've lost some of 
the driver debug messages as it seems QEMU debug messages overwrite those 
when both are directed to stdout so I can't capture them both but added 
comments to explain what stage they are at.

> > > (background on emulated machine:
> > > https://osdn.net/projects/qmiga/wiki/SubprojectPegasos2 )

(There's also board schematics linked from that page in case someone is 
interested about how IRQs are connected, it's not impossible I've got that 
wrong in my model as I don't really know what real hardware does. The 
work in progress patches implementing pegasos2 are in the git branch also 
linked from above.)

> > > Originally I had both Linux and MorphOS fail after enabling BMDMA
> > > before I had interrupt controller emulation (so that means it could be
> > > a problem with that in your case as well so something to check). Now
> > > that I've implemented interrupts Linux boots with DMA from CDROM but
> > > MorphOS is still not happy.
> >
> > I could now also reproduce the same with AmigaOS4 on pegasos2 where I
> > got same missing interrupt problem:
> >
> > ---> Port 1
> > IOBase 00001010, AltBase 0000101E
> > bmcr_base 00001028
> > MMIOBase 00000000
> > Config not forced, scanning ...
> > 1 device(s) on port
> >  0 Master : 'ATAPI'
>
> "unit 2", I assume
>
> >  1 Slave  : 'unknown type'
>
> "unit 3", I assume

Yes, there was also a "---> Port 0" before that, checking for primary 
master and slave devices that were unit 0 and 1 but I did not include that 
for brevity. Thus secondary master and slave devices are numbered 2 and 3.

> > So there's definitely a problem with interrupts but not sure where. Also
> > don't know why it detects an unknown slave device which then it decides
> > is invalid. Maybe this is normal on an IDE bus with only one device or
> > is it a problem in emulation?
>
> I don't actually know myself. We *do* always have two IDEState objects
> per port, but maybe we're letting some unknown state sneak through --
> filling in a register improperly, perhaps?

To make it clear, the problem I have that prevents it from working is the 
missing interrupt after switching to UDMA. The unknown slave device is 
only causing a delay in detection but then it determines that the 
secondary device is not working and goes further so I've only reported 
that as a potential problem in emulation that could be investigated and 
fixed but it's not the main problem that prevents working.

> It's probably not ide_ioport_read -- but I see complaints about the
> reset signature too, so maybe we've gotten that wrong.
>
> You can look at ide_set_signature to see how we set the drive
> signatures; called from ide_reset (and many other places too)

Unfortunately I don't know much about IDE so likely I would not find 
out much there.

> I think ide_init_drive is only meant to be called on devices that
> actually exist and are plugged in. It initializes drive_kind to one of
> IDE_HD, IDE_CD, or IDE_CFATA -- empty or missing isn't an option here.
>
> (Hm, this means it defaults to IDE_HD actually.)

> From the logs I've seen it does seem to expect an ATA device i.e. HDD as 
unit 3 but after trying to identify it it gets an error and finds that 
there's no working device there and goes with just the CD drive. MorphOS 
does the same but trying a few times before giving up, there are some logs 
showing that in the above ticket,

> The tricky thing is that IDEState belongs to the parent bus -- not the
> drive object itself -- and the bus always has two slots.
>
> We select between the two by setting bus->unit; and we don't appear to
> do any kind of actual guarding that the drive actually exists.
>
> (I suppose guests are free to issue commands to non-existant drives if
> they want to, but they're not going to be able to perform work.)
>
> ...but ide_reset_bus calls ide_reset on both slots regardless of the
> presence of a device or not.

I remember seeing log messages from guest that said it reset the ide bus 
before trying to probe to avoid hangs that probably happen on real 
hardware but I don't find them now. But it's possible there was a bus 
reset before the above probing. I think it wants to get the bus to a known 
state to avoid problems due to firmware or something else leaving the bus 
and devices in some weird state.

> (This is probably just a side effect of the interrupt getting lost and
> having the guest try to reset the controller, then noticing weird state
> after the reset.)
>
> It sounds like the real problem is either in the bmdma controller (or
> its unique interaction with hw/ide/core.c -- which is possible) or in
> the interrupt routing somewhere else.

Despite being able to reproduce similar problems on ppc mips and sparc64 
emulations with two different ide controllers (sharing bmdma code) it 
could well be that all of these have some kind of IRQ routing problem as 
neither of these are finished and stable yet so I'm not sure the missing 
IRQ is coming from IDE code. Especially because in the sii3112 case it 
seems to be generated and only get lost in the last step in the IRQ 
conroller but don't see why. The guest should have enabled the 
corresponding interrupt I think. For via-ide it's not obvious where the 
IRQ gets lost, it seems to be generated later so I don't understand that 
completely either.

> If you have any IDE traces from a hang, feel free to throw them up on a
> pastebin for me to take a peek at; it might help for me to see the exact
> sequence that causes a hang in QEMU's IDE terms to see if I can't
> "reverse engineer" what the guest is hoping to have happen. Maybe I can
> trace this to a bad register value.

See more logs in the ticket, let me know if you'd need me to check 
something else. Thanks a lot for looking into this, I'd need to look up 
all of these IDE interactions to try to make sense of the logs which would 
take too much time. Maybe you can spot something easier.

> (Hm, it's failing on pio_in? It's using PIO on an IDE drive with a DMA
> controller? Is it failing to enable DMA and then failing to use PIO as a
> backup too? Maybe there are two bugs.)

Before getting to this point where it hangs waiting for the IRQ after 
switching on UDMA the firmware did find all devices and loaded the OS from 
the CD all right and the OS also can talk to ide devices as long as it 
uses only PIO. The IRQ problem only happens after DMA is enabled but I'm 
not sure it uses IRQ in PIO at all.

AmigaOS and MorphOS really want to use UDMA only (or highest DMA mode the 
device reports) as otherwise it would be too slow on real hardware so I 
don't think it wants to fall back, just failing the drive if it does not 
work in the DMA mode it reports.

Linux may do fallback I'm not sure but booting from a 
debian-8.11.0-powerpc-netinst.iso seems to work even after it says 
enabling DMA, whereas I got the same problem with Linux before I hooked up 
IRQs correctly. Linux logs I get now with via-ide:

[    0.858497] pata_via 0000:00:0c.1: version 0.3.4

[    0.905257] scsi0 : pata_via

[    0.925207] scsi1 : pata_via
[    0.925722] ata1: PATA max UDMA/100 cmd 0x1000 ctl 0x100c bmdma 0x1020 irq 9
[    0.925906] ata2: PATA max UDMA/100 cmd 0x1010 ctl 0x101c bmdma 0x1028 irq 9

[    1.297537] ata2.01: NODEV after polling detection
[    1.298650] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
[    1.318205] ata2.00: configured for UDMA/100
[    1.330427] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ 
PQ: 0 ANSI: 5
[    1.401652] sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[    1.401851] cdrom: Uniform CD-ROM driver Revision: 3.20
[    1.404116] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    1.410853] sr 1:0:0:0: Attached scsi generic sg0 type 5

And if boots all right. I think Linux can use DMA and I see interrupts 
with Linux so maybe the drivers work differently and only the 
AmigaOS/MorphOS drivers do something that does not work with QEMU or 
depend on some features on hardware that I did not model correctly.

Regards,
BALATON Zoltan