Re: [PATCH QEMU v8 4/9] migration: Introduce dirty-limit capability

[Markus Armbruster]

Yong Huang <yong.huang@smartx.com> writes:

> On Tue, Jul 18, 2023 at 7:04 PM Markus Armbruster <armbru@redhat.com> wrote:
>
>> Yong Huang <yong.huang@smartx.com> writes:
>>
>> > On Thu, Jul 13, 2023 at 8:44 PM Markus Armbruster <armbru@redhat.com>
>> wrote:
>> >
>> >> ~hyman <hyman@git.sr.ht> writes:
>> >>
>> >> > From: Hyman Huang(黄勇) <yong.huang@smartx.com>
>> >> >
>> >> > Introduce migration dirty-limit capability, which can
>> >> > be turned on before live migration and limit dirty
>> >> > page rate durty live migration.
>> >> >
>> >> > Introduce migrate_dirty_limit function to help check
>> >> > if dirty-limit capability enabled during live migration.
>> >> >
>> >> > Meanwhile, refactor vcpu_dirty_rate_stat_collect
>> >> > so that period can be configured instead of hardcoded.
>> >> >
>> >> > dirty-limit capability is kind of like auto-converge
>> >> > but using dirty limit instead of traditional cpu-throttle
>> >> > to throttle guest down. To enable this feature, turn on
>> >> > the dirty-limit capability before live migration using
>> >> > migrate-set-capabilities, and set the parameters
>> >> > "x-vcpu-dirty-limit-period", "vcpu-dirty-limit" suitably
>> >> > to speed up convergence.
>> >> >
>> >> > Signed-off-by: Hyman Huang(黄勇) <yong.huang@smartx.com>
>> >> > Acked-by: Peter Xu <peterx@redhat.com>
>> >> > Reviewed-by: Juan Quintela <quintela@redhat.com>
>> >>
>> >> [...]
>> >>
>> >> > diff --git a/qapi/migration.json b/qapi/migration.json
>> >> > index e43371955a..031832cde5 100644
>> >> > --- a/qapi/migration.json
>> >> > +++ b/qapi/migration.json
>> >> > @@ -497,6 +497,15 @@
>> >> >  #     are present.  'return-path' capability must be enabled to use
>> >> >  #     it.  (since 8.1)
>> >> >  #
>> >> > +# @dirty-limit: If enabled, migration will use the dirty-limit
>> >> > +#     algorithm to throttle down guest instead of auto-converge
>> >> > +#     algorithm. This algorithm only works when vCPU's dirtyrate
>> >>
>> >> Two spaces after sentence-ending punctuation, please.
>> >>
>> >> "dirty rate" with a space, because that's how we spell it elsewhere.
>> >>
>> >> > +#     greater than 'vcpu-dirty-limit', read processes in guest os
>> >> > +#     aren't penalized any more, so the algorithm can improve
>> >> > +#     performance of vCPU during live migration. This is an optional
>> >> > +#     performance feature and should not affect the correctness of
>> the
>> >> > +#     existing auto-converge algorithm. (since 8.1)
>> >> > +#
>> >>
>> >> I'm still confused.
>> >>
>> >> The text suggests there are two separate algorithms "to throttle down
>> >> guest": "auto converge" and "dirty limit", and we get to pick one.
>> >> Correct?
>> >>
>> > Yes, indeed !
>> >
>> >>
>> >> If it is correct, then the last sentence feels redundant: picking
>> >> another algorithm can't affect the algorithm we're *not* using.  What
>> >> are you trying to express here?
>> >>
>> > What i want to express is that the new algorithm implementation does
>> > not affect the original algorithm, leaving it in the comments seems
>> > redundant indeed.  I'll drop this in the next version.
>>
>> Works for me.
>>
>> >> When do we use "auto converge", and when do we use "dirty limit"?
>> >>
>> >> What does the user really need to know about these algorithms?  Enough
>> >> to pick one, I guess.  That means advantages and disadvantages of the
>> >> two algorithms.  Which are?
>> >
>> > 1. The implementation of dirty-limit is based on dirty-ring, which is
>> > qualified
>> >    to big systems with huge memories and can improve huge guest VM
>> >     responsiveness remarkably during live migration. As a consequence,
>> > dirty-limit
>> >     is recommended on platforms with huge guest VMs as is the way with
>> > dirty-ring.
>> > 2. dirty-limit convergence algorithm does not affect the "read-process"
>> in
>> > guest
>> >    VM, so guest VM gains the equal read performance nearly as it runs on
>> > host
>> >    during the live migration. As a result, dirty-limit is recommended if
>> > the guest
>> >     VM requires a stable read performance.
>> > The above explanation is about the recommendation of dirty-limit, please
>> > review,
>> > if it's ok, i'll place it in the comment of the dirty-limit capability.
>>
>> Yes, please.  But before that, I have still more questions.  "This
>> algorithm only works when vCPU's dirtyrate greater than
>> 'vcpu-dirty-limit'" is a condition: "FEATURE only works when CONDITION".
>>
> I failed to express my meaning again : ( .  "Throttle algo only works when
> vCPU's  dirtyrate greater than 'vcpu-dirty-limit' " should change to
> "vCPU throttle only works when vCPU's dirtyrate greater than
> 'vcpu-dirty-limit'".
> Not the whole "algo" !

Let me paraphrase to make sure I got it...  The vCPU is throttled as
needed to keep its dirty rate within the limit set with
set-vcpu-dirty-limit.  Correct?

What happens when I enable the dirty limit convergence algorithm without
setting a limit with set-vcpu-dirty-limit?

>> What happens when the condition is not met?  How can the user ensure the
>> condition is met?
>>
>> [...]
>>
>>