Re: [PATCH v4] i386: Add ratelimit for bus locks acquired in guest

[Dr. David Alan Gilbert]

* Chenyi Qiang (chenyi.qiang@intel.com) wrote:
> A bus lock is acquired through either split locked access to writeback
> (WB) memory or any locked access to non-WB memory. It is typically >1000
> cycles slower than an atomic operation within a cache and can also
> disrupts performance on other cores.
> 
> Virtual Machines can exploit bus locks to degrade the performance of
> system. To address this kind of performance DOS attack coming from the
> VMs, bus lock VM exit is introduced in KVM and it can report the bus
> locks detected in guest. If enabled in KVM, it would exit to the
> userspace to let the user enforce throttling policies once bus locks
> acquired in VMs.
> 
> The availability of bus lock VM exit can be detected through the
> KVM_CAP_X86_BUS_LOCK_EXIT. The returned bitmap contains the potential
> policies supported by KVM. The field KVM_BUS_LOCK_DETECTION_EXIT in
> bitmap is the only supported strategy at present. It indicates that KVM
> will exit to userspace to handle the bus locks.
> 
> This patch adds a ratelimit on the bus locks acquired in guest as a
> mitigation policy.
> 
> Introduce a new field "bus_lock_ratelimit" to record the limited speed
> of bus locks in the target VM. The user can specify it through the
> "bus-lock-ratelimit" as a machine property. In current implementation,
> the default value of the speed is 0 per second, which means no
> restrictions on the bus locks.
> 
> As for ratelimit on detected bus locks, simply set the ratelimit
> interval to 1s and restrict the quota of bus lock occurence to the value
> of "bus_lock_ratelimit". A potential alternative is to introduce the
> time slice as a property which can help the user achieve more precise
> control.
> 
> The detail of bus lock VM exit can be found in spec:
> https://software.intel.com/content/www/us/en/develop/download/intel-architecture-instruction-set-extensions-programming-reference.html
> 
> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>

Hi Chenyi,

  I noticed in this patch:


> +static void kvm_rate_limit_on_bus_lock(void)
> +{
> +    uint64_t delay_ns = ratelimit_calculate_delay(&bus_lock_ratelimit_ctrl, 
> 1);
> +
> +    if (delay_ns) {
> +        g_usleep(delay_ns / SCALE_US);
> +    }
> +}

and wondered if this would block cpu kicks, and what would happen if
delay_ns got quite big - Eduardo thinks it might get upto 1s.

Also, it feels similar to what migration does during 'auto converge';
see softmuu/cpu-throttle.c - instead of doing your own g_usleep
you could call cpu_throttle_set with a given throttle rate.

Dave

> +
>  MemTxAttrs kvm_arch_post_run(CPUState *cpu, struct kvm_run *run)
>  {
>      X86CPU *x86_cpu = X86_CPU(cpu);
> @@ -4237,6 +4271,9 @@ MemTxAttrs kvm_arch_post_run(CPUState *cpu, struct 
> kvm_run *run)
>      } else {
>          env->eflags &= ~IF_MASK;
>      }
> +    if (run->flags & KVM_RUN_X86_BUS_LOCK) {
> +        kvm_rate_limit_on_bus_lock();
> +    }
>  
>      /* We need to protect the apic state against concurrent accesses from
>       * different threads in case the userspace irqchip is used. */
> @@ -4595,6 +4632,10 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run 
> *run)
>          ioapic_eoi_broadcast(run->eoi.vector);
>          ret = 0;
>          break;
> +    case KVM_EXIT_X86_BUS_LOCK:
> +        /* already handled in kvm_arch_post_run */
> +        ret = 0;
> +        break;
>      default:
>          fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
>          ret = -1;
> -- 
> 2.17.1
> 
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK