[Qemu-devel] ARM Cortex-M issues

[Bill Paul]

I recently started tinkering with ChibiOS as part of a small personal project 
and wanted to test some of the demo configurations that it has with a machine 
simulator before going all in on a reference board. I decided to try QEMU 
2.6.0. I was mainly interested in an ARM Cortex-M machine, so I figured I 
would try the Netduino 2 or Stellaris board models.

Unfortunately it's been a frustrating experience because there seem to be 
several key places where QEMU's hardware emulation diverges from reality. The 
ChibiOS examples often seem to depend on behavior that is valid for actual 
hardware but which is either broken or just missing in QEMU. Some of these 
issues are board-specific, but the last one seems a bit more general.

The Netduino 2 is built using an ST Micro STM32F205 chip. This is a Cortex-M3 
with ST Micro-specific peripherals. ChibiOS has an example for the STM32F207 
which I figured was close enough to try.

The first problem is that the stm32f2xx_usart.c does not fully emulate the 
hardware. In particular it's missing support for the TX fifo empty state 
(TXE). There is no definition for the TXEIE bit and the logic for handling the 
TXE interrupt is broken. ChibiOS waits to see when the TXE bit is set before 
writing a character out; it's basically waiting to make sure the transmitter 
is ready before sending any new data. Since this logic is not implemented 
correctly, ChibiOS stalls when trying to output anything to the serial port.

I managed to hack my way around that, but then I ran into the second problem: 
the stm32f2xx_timer.c also doesn't fully emulate the hardware. It's missing 
support for output compare mode, which ChibiOS also depends on. Unfortunately 
I wasn't quite able to fix this myself because the ST Micro documentation 
concerning the STM32F2xx timer implementation is a bit hard to follow, so it 
wasn't clear to me just what the correct logic should be. (Implementing timers 
in QEMU seems a bit complicated too.) I got it to limp along, but couldn't 
quite get it exactly right. I actually got to the point where the ChibiOS 
tests would run, but not reliably.
 
So I gave up on the STM32F2xx model and decided to try just the generic 
Cortex-M mode. This example uses just the systick counter which is part of the 
system control block (SCB) in the Cortex-M core and runs a couple of simple 
threads that just wake up to count seconds and minutes.

When I tried to run this example with the Netduino 2 machine, QEMU crashed and 
dumped core as soon as the example finished initializing the systick 
registers.

After some research, I discovered the following:

https://bugs.launchpad.net/qemu/+bug/696094

Note that it mentions the system_clock_scale global variable in armv7m_nvic.c 
not being initialized. However this bug specifically references the Stellaris 
lm3s811evb board mode. It seems this has been fixed for the Stellaris machine 
simulation, but not for the Netduino 2.

So I switched to the Stellaris lm3s811evb machine model for my testing. Now 
QEMU wouldn't dump core anymore, but ChibiOS would hang after only one timer 
interrupt. I found it was stuck in _port_exit_from_isr.

Apparently what happens here is that ChibiOS/RT is using an interesting 
preemption model where it uses a software-triggered non-maskable interrupt to 
trigger a context switch. Prior to calling _port_exit_from_isr, ChibiOS locks 
out interrupts (wuth a "cspid i" instruction), which the comments indicate is 
to ensure atomic operation. The _port_exit_from_isr then triggers a software 
NMI by writing to the ICSR register in the SCB to set the NMIPENDSET bit, and 
then it falls through to an infinite loop. The expectation of course is that 
the NMI handler will run. but it doesn't. Instead ChibiOS just stalls forever 
at the loop instruction.

Apparently the simulation doesn't honor the notion that the non-maskable 
interrupt should be non-maskable. The NMI interrupt is handled by arm_gic.c, 
which calls qemu_set_irq(), however the target-arm/cpu.c code ignores it 
because the PSTATE_I bit is not set in the CPU state. Yes, we have executed a 
"cspid i" instruction to mask interrupts, but no that shouldn't prevent the 
NMI from being delivered.

This problem in particular has me baffled, and I'm forced to ask questions. It 
seems like the ARM CPU simulation doesn't support the notion of NMI at all. (I 
think it would be a separate input from IRQ and FIQ, no?) How is this supposed 
to work? It seems like the Cortex-M hardware allows for software-generated 
NMIs, so why isn't this supported? How is this supposed to work on other ARM 
CPUs like Cortex-A? Is it really the case that nobody has tried this before?

ChibiOS/RT seems to support an alternate preemption scheme that uses the 
PendSV interrupt instead of NMI, and if I compile it to use that mode, then 
the example works with the Stellaris machine model. That feels like a hack 
though: shouldn't a software NMI just work?

-Bill

-- 
=============================================================================
-Bill Paul            (510) 749-2329 | Senior Member of Technical Staff,
                 address@hidden | Master of Unix-Fu - Wind River Systems
=============================================================================
   "I put a dollar in a change machine. Nothing changed." - George Carlin
=============================================================================