Re: [PATCH v6 3/3] target/ppc: support single stepping with KVM HV

[David Gibson]

On Tue, Jan 21, 2020 at 05:23:03PM -0300, Fabiano Rosas wrote:
> David Gibson <address@hidden> writes:
> 
> (...)
> >> > Hrm.... I don't actually see how changing env->msr helps you here.
> >> > AFAICT if kvm_insert_breakpoint() resolves to kvm_arch_sw_breakpoint()
> >> > it doesn't rely on the MSR value at all.  If it resolves to
> >> > kvm_arch_hw_breakpoint(), then it looks like it just stashes
> >> > information to be pushed into KVM when we re-enter the guest.  None of
> >> > the information stashed appears to depend on the current MSR, and once
> >> > we re-enter the MSR will already have been restored.
> >> >
> >>
> >> This is the call chain:
> >> 
> >> kvm_arch_insert_sw_breakpoint -> cpu_memory_rw_debug ->
> >> cpu_get_phys_page_attrs_debug -> ppc_cpu_get_phys_page_debug ->
> >> ppc64_v3_get_phys_page_debug -> ppc_radix64_get_phys_page_debug:
> >> 
> >>     /* Handle Real Mode */
> >>     if (msr_dr == 0) {
> >>         /* In real mode top 4 effective addr bits (mostly) ignored */
> >>         return eaddr & 0x0FFFFFFFFFFFFFFFULL;
> >>     }
> >
> > Ah, right.  Basically the issue is that kvm_insert_breakpoint() takes
> > an effective address, not a real address, but it might be happening in
> > a different context than we're executing right now.
> >
> > Ok, that makes sense.  Though.. aren't you always inserting the
> > breakpoint into an interrupt vector?  So wouldn't it always be MMU
> > off?  Under what circumstances would this get called with mmu_on =
> > true?
> 
> Well, the MSR state at the moment of the breakpoint is that of the
> currently executing instruction.

Uh... at the moment of setting the breakpoint, or the moment of
hitting the breakpoint.

> So this gets called with mmu_on = true
> very often because we're often debugging code than runs with
> IR|DR=1.

Uh... but isn't the whole point here that the state of mmu_on might
not match the MSR state.  So the two sentences above don't seem to
mesh together.

What I think I'm understanding from the code is that in order to *set*
the breakpoint, you need to set up the MSR to match what you expect it
will be when you *hit* the breakpoint.  Yes?

But since the breakpoint is always placed in an interrupt vector,
won't that always be real mode?  Or is this one of the vectors that
can be entered in virtual mode on recent chips?

> However, we could be at a point when IR|DR=1, but the next traced
> instruction will execute with IR|DR=0. This happens at the rfid at the
> end of __enter_rtas, for instance.
> 
> So ppc_radix64_get_phys_page_debug will check the MSR, see that we are
> (now) not in real mode and proceed with the page table walk, which could
> fail.
> 
> In the particular case of the __enter_rtas rfid, we have PIDR=1 [1] so
> if we don't exit ppc_radix64_get_phys_page_debug at the msr_dr == 0
> check, it will fail to translate the address.
> 
> 1 - 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=eeb715c3e995fbdda0cc05e61216c6c5609bce66
> 
> >> Actually, I think there is a bug after ppc_cpu_get_phys_page_debug
> >> somewhere. There are some cases where GDB wants to read/write to some
> >> memory, but it gets denied access. Presumably because of one such
> >> discrepancy as the one above. I need to spend more time looking at this
> >> to define the problem properly, though.
> >
> > Hm, ok.
> >
> >> >> +    /*
> >> >> +     * MSR_SE = 1 will cause a Trace Interrupt in the guest after the
> >> >> +     * next instruction executes. If this is a rfid, use SRR1 instead
> >> >> +     * of MSR.
> >> >> +     */
> >> >> +    if (rfid) {
> >> >> +        if ((env->spr[SPR_SRR1] >> MSR_SE) & 1) {
> >> >> +            /*
> >> >> +             * The guest is doing a single step itself. Make sure we
> >> >> +             * restore it later.
> >> >> +             */
> >> >> +            env->sstep_kind = SSTEP_GUEST;
> >> >> +        }
> >> >> +
> >> >> +        env->spr[SPR_SRR1] |= (1ULL << MSR_SE);
> >> >> +        mmu_on = srr1_ir & srr1_dr;
> >> >
> >> > s/&/&&/
> >> >
> >> 
> >> Ack.
> >> 
> >> >> +    } else {
> >> >> +        env->msr |= (1ULL << MSR_SE);
> >> >> +        mmu_on = msr_ir & msr_dr;
> >> >
> >> > s/&/&&/
> >> >
> >> 
> >> Ack.
> >> 
> >> > Also, what happens if the guest is using MSR[DR] != MSR[IR]?  It's
> >> > rare, but it is occasionally useful.
> >> 
> >> I understand from the ISA that for the purposes of AIL, both bits need
> >> to be set. So mmu_on = 0 is correct here.
> >
> > I'm not sure what you mean by "for the purposes of AIL".
> >
> 
> The reason I'm tracking the translation state here is to be able to tell
> what will be the value of AIL, since an Alternate Interrupt Location is
> not used when translation is disabled. In the ISA, under "Alternate
> Interrupt Location" the only mention of MSR_IR != MSR_DR is:
> 
> "Other interrupts that occur when MSR IR=0 or MSR DR=0, are taken as
> if LPCR AIL=0."
> 
> and my interpretation of that text is that AIL value is 0 when IR DR are
> either 0b00, 0b01 or 0b10.
> 
> So "for the purposes of AIL", I'm considering either IR=0 or DR=0 as
> meaning MMU off.

But AFAICT the mmu_on flag you're setting here has influences other
than tracking AIL.

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature