Re: [Qemu-devel] ARM QEMU/KVM and TrustZone

[Christoffer Dall]

> On 22 May 2012 13:22, Peter Maydell <address@hidden> wrote:
>> Historically for QEMU we haven't implemented TrustZone support even
>> though we claim to emulate CPUs that provide it. Instead we provide a
>> CPU which mostly looks like a variant of the real thing without the
>> TrustZone feature. We then bolt on a few extra cp15 registers (eg the
>> SCR) as a pragmatic move to get Linux guests to run. Now we're also
>> dealing with KVM on ARM I'd like to define things a bit more solidly
>> so KVM and TCG agree on what the CPU model they present is.
>>
>> There are several possible environments we could provide
>> to a guest:
>>  (1) a CPU with full TrustZone support
>>  (2) a CPU without TrustZone at all
>>  (3) a TZ CPU running in NonSecure PL0/PL1
>>  (4) a TZ CPU running in Secure PL0/PL1
>>
>> In some ways (1) is the "purist" solution -- emulate exactly what the
>> hardware does. However:
>>
>>  * on TCG it would require a lot of work, including new functionality
>>   in core QEMU (to support having different CPU cores being able to
>>   see different views of memory, and having the S/NS attribute
>>   attached to memory transactions)
>>
>>  * it isn't possible in KVM, because the ARM Virtualization Extensions
>>   don't allow you to fake the CPSR a guest sees, and so you can't
>>   make the guest believe it is in Monitor mode
>>
>> Option (2) is architecturally sanctioned (ie TrustZone is an optional
>> feature, not mandatory), but it doesn't correspond to real CPUs, in
>> that the hardware Cortex-A8/A9/A15 always have TrustZone. So we're
>> modelling something that doesn't really exist.
>>
>> Options (3) and (4) correspond to the environment an OS guest
>> typically actually uses on hardware. For ARM's devboards (versatile
>> express etc) Linux runs in the Secure world but it doesn't actually
>> use any of the TrustZone functionality, it's just a "give me full
>> access to everything" setup. For just about every other ARM system,
>> the boot rom or equivalent keeps Secure world to itself, and the OS
>> kernel runs in the NonSecure world. (This typically means that the
>> boot rom provides a set of board-specific entry points via the Secure
>> Monitor Call (SMC) instruction for doing operations like "invalidate
>> whole L2 cache" which require secure privileges.)
>>

Is there anything preventing people from writing a small bootloader
that switches into non-secure mode and runs kernels there as a general
approach (apart from laziness)?

>> Proposal:
>>
>> My suggestion is that we present the guest with a view that looks like
>> a sort of superset of (2) (3) and (4), ie sufficient that a guest
>> expecting any of those environments can run. In particular:
>>
>>  * no cp15 registers have secure/nonsecure banking
>>  * there is only one memory space visible
>>  * secure-access-only permissions are not enforced
>>  * the handful of only-in-trustzone registers are implemented
>>   (eg VBAR, MVBAR)
>>  * we implement a "fake monitor mode"
>>
>> The aim of the "fake monitor mode" is to allow us to provide fake
>> qemu-specific bootroms which implement whatever the board's SMC
>> interface is, without having to write specific KVM kernel code for
>> each board. So we don't have to run arbitrary secure-world guest code.
>> The rules are:
>>  * on an SMC instruction we enter the guest at the SMC vector
>>   as defined by the MVBAR (monitor vector base address register)
>>  * we actually run with the same access permissions as above
>>   (and under KVM if you look at CPSR.M it will tell you you're
>>   in Supervisor mode)
>>  * return from the SMC is via a standard exception return insn
>>  * we don't implement the separate memory space for the secure
>>   world. (This implies that you need to find space in the
>>   non-secure world's physical memory map for the bootrom shim;
>>   not a big deal I think since we already have a requirement
>>   for some space to put QEMU's arm_boot trivial bootloader.)
>>

you could have a separate set of stage-2 translation tables for this
and keep things separate for real (or would we rely on fake-SMC code
to directly be able to read fake-non-secure data, which is still
possible through a different memory map I guess).

>> The code written for this fake monitor mode environment is likely to
>> be able to work OK if we ever implement full TrustZone support in TCG
>> QEMU.
>>

what kind of operations would be required from SMC calls in a KVM
guest setting? I can see this in an embedded market, but are they not
likely to even capture Hyp mode already and set things up as required?
What I mean is, if KVM is currently targeting Calxeda-type setups will
we ever run kernels that require SMC operations as guests?

It feels a bit premature to implement all this.

>> Work required:
>>
>> * Documentation: the general principles as listed above
>> * TCG: make sure we have implementations of all the TZ registers
>> * TCG: implement the SMC and fake-monitor-mode
>>  (I already have patches from Nokia in the qemu-linaro
>>  stack which can be cleaned up and used here)
>> * KVM: implement emulation of MVBAR
>> * KVM: set the config bit so SMC is trapped to the hypervisor
>>  and causes guest restart at the right entrypoint
>> * KVM: if there turns out to be anything that fake-monitor-mode
>>  needs to do that requires Hyp privilege we'd need a hypercall
>>  ABI, but I can't currently think of anything
>>
>> I think that's basically a fairly small set of work to formalise
>> the approach we're already taking in practice, and make it a little
>> more flexible.
>>
>> Opinions?
>>

It feels like quite a bit of complexity at this point. But if we need
it, we need it. I'm just not convinced of that yet.

-Christoffer