Re: [Qemu-ppc] [Qemu-devel] [PULL 57/60] target/ppc: add HV support for POWER9

[David Gibson]

On Wed, Mar 13, 2019 at 01:32:00PM +1100, David Gibson wrote:
> On Tue, Mar 12, 2019 at 11:01:15AM -0400, Cleber Rosa wrote:
> > On Sun, Mar 10, 2019 at 07:27:00PM +1100, David Gibson wrote:
> > > From: Cédric Le Goater <address@hidden>
> > > 
> > > We now have enough support to boot a PowerNV machine with a POWER9
> > > processor. Allow HV mode on POWER9.
> > > 
> > > Signed-off-by: Cédric Le Goater <address@hidden>
> > > Message-Id: <address@hidden>
> > > Signed-off-by: David Gibson <address@hidden>
> > > ---
> > >  target/ppc/translate_init.inc.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/target/ppc/translate_init.inc.c 
> > > b/target/ppc/translate_init.inc.c
> > > index af70a3b78c..0bd555eb19 100644
> > > --- a/target/ppc/translate_init.inc.c
> > > +++ b/target/ppc/translate_init.inc.c
> > > @@ -8895,7 +8895,7 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
> > >                         PPC_CACHE | PPC_CACHE_ICBI | PPC_CACHE_DCBZ |
> > >                         PPC_MEM_SYNC | PPC_MEM_EIEIO |
> > >                         PPC_MEM_TLBSYNC |
> > > -                       PPC_64B | PPC_64BX | PPC_ALTIVEC |
> > > +                       PPC_64B | PPC_64H | PPC_64BX | PPC_ALTIVEC |
> > >                         PPC_SEGMENT_64B | PPC_SLBI |
> > >                         PPC_POPCNTB | PPC_POPCNTWD |
> > >                         PPC_CILDST;
> > > @@ -8907,6 +8907,7 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
> > >                          PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
> > >                          PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL;
> > >      pcc->msr_mask = (1ull << MSR_SF) |
> > > +                    (1ull << MSR_SHV) |
> > >                      (1ull << MSR_TM) |
> > >                      (1ull << MSR_VR) |
> > >                      (1ull << MSR_VSX) |
> > 
> > This change prevents a Fedora 29 kernel[1] from booting... is this
> > intended or a known limitation of the Fedora 29 kernel?
> > 
> > FIY, by using a "power8" CPU the Fedora 29 kernel boots successfully.
> > For a reproducer, please refer to [2].
> 
> Ouch.  I don't know how, but this has introduced a nasty regression.
> I've reproduced it myself, here's what I've observed so far.  Nothing
> very surprising, but helps narrow it down a bit.
> 
>  * It works with KVM
>  * It works with -cpu power8
>  * It also fails with the current RHEL8 kernel
>  * It also fails on a ppc64le host (using TCG)
>  * Also fails with a fresh build upstream kernel
> 
> It's very strange because a) the flags it sets are already enabled for
> power8, which seems to work and b) literally the only thing that
> PPC_64H seems to control is presence of the hrfid instruction, which
> the guest kernel shouldn't be even attempting to use.
> 
> It seems to be hitting an explicit panic() in the guest code (no
> message because it's before we have a working console).
> 
> Continuing to investigate...

Ok, pretty sure I found it.  This patch exposed, via a roundabout
route a rather older bug.  A hypercall wasn't properly setting the
LPCR[GTSE] bit which meant now that hypervisor support was turned on,
the guest kernel got an early exception because it tried to execute a
tlbie instruction which is hypervisor privileged without LPCR[GTSE].

I've posted a fix and added it to ppc-for-4.0.

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature