Re: [PATCH RFC 13/15] migration: Add postcopy-preempt capability

[Dr. David Alan Gilbert]

* Peter Xu (peterx@redhat.com) wrote:
> Firstly, postcopy already preempts precopy due to the fact that we do
> unqueue_page() first before looking into dirty bits.
> 
> However that's not enough, e.g., when there're host huge page enabled, when
> sending a precopy huge page, a postcopy request needs to wait until the whole
> huge page that is sending to finish.  That could introduce quite some delay,
> the bigger the huge page is the larger delay it'll bring.
> 
> This patch adds a new capability to allow postcopy requests to preempt 
> existing
> precopy page during sending a huge page, so that postcopy requests can be
> serviced even faster.
> 
> Meanwhile to send it even faster, bypass the precopy stream by providing a
> standalone postcopy socket for sending requested pages.
> 
> Since the new behavior will not be compatible with the old behavior, this will
> not be the default, it's enabled only when the new capability is set on both
> src/dst QEMUs.
> 
> This patch only adds the capability itself, the logic will be added in follow
> up patches.
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  migration/migration.c | 23 +++++++++++++++++++++++
>  migration/migration.h |  1 +
>  qapi/migration.json   |  8 +++++++-
>  3 files changed, 31 insertions(+), 1 deletion(-)
> 
> diff --git a/migration/migration.c b/migration/migration.c
> index 15a48b548a..84a8fbd80d 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1227,6 +1227,11 @@ static bool migrate_caps_check(bool *cap_list,
>              error_setg(errp, "Postcopy is not compatible with 
> ignore-shared");
>              return false;
>          }
> +
> +        if (cap_list[MIGRATION_CAPABILITY_MULTIFD]) {
> +            error_setg(errp, "Multifd is not supported in postcopy");
> +            return false;
> +        }
>      }
>  
>      if (cap_list[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) {
> @@ -1270,6 +1275,13 @@ static bool migrate_caps_check(bool *cap_list,
>          return false;
>      }
>  
> +    if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]) {
> +        if (!cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) {
> +            error_setg(errp, "Postcopy preempt requires postcopy-ram");
> +            return false;
> +        }
> +    }
> +
>      return true;
>  }
>  
> @@ -2623,6 +2635,15 @@ bool migrate_background_snapshot(void)
>      return s->enabled_capabilities[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT];
>  }
>  
> +bool migrate_postcopy_preempt(void)
> +{
> +    MigrationState *s;
> +
> +    s = migrate_get_current();
> +
> +    return s->enabled_capabilities[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT];
> +}
> +
>  /* migration thread support */
>  /*
>   * Something bad happened to the RP stream, mark an error
> @@ -4239,6 +4260,8 @@ static Property migration_properties[] = {
>      DEFINE_PROP_MIG_CAP("x-compress", MIGRATION_CAPABILITY_COMPRESS),
>      DEFINE_PROP_MIG_CAP("x-events", MIGRATION_CAPABILITY_EVENTS),
>      DEFINE_PROP_MIG_CAP("x-postcopy-ram", MIGRATION_CAPABILITY_POSTCOPY_RAM),
> +    DEFINE_PROP_MIG_CAP("x-postcopy-preempt",
> +                        MIGRATION_CAPABILITY_POSTCOPY_PREEMPT),
>      DEFINE_PROP_MIG_CAP("x-colo", MIGRATION_CAPABILITY_X_COLO),
>      DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM),
>      DEFINE_PROP_MIG_CAP("x-block", MIGRATION_CAPABILITY_BLOCK),
> diff --git a/migration/migration.h b/migration/migration.h
> index d0c0902ec9..9d39ccfcf5 100644
> --- a/migration/migration.h
> +++ b/migration/migration.h
> @@ -391,6 +391,7 @@ int migrate_decompress_threads(void);
>  bool migrate_use_events(void);
>  bool migrate_postcopy_blocktime(void);
>  bool migrate_background_snapshot(void);
> +bool migrate_postcopy_preempt(void);
>  
>  /* Sending on the return path - generic and then for each message type */
>  void migrate_send_rp_shut(MigrationIncomingState *mis,
> diff --git a/qapi/migration.json b/qapi/migration.json
> index bbfd48cf0b..f00b365bd5 100644
> --- a/qapi/migration.json
> +++ b/qapi/migration.json
> @@ -452,6 +452,12 @@
>  #                       procedure starts. The VM RAM is saved with running 
> VM.
>  #                       (since 6.0)
>  #
> +# @postcopy-preempt: If enabled, the migration process will allow postcopy
> +#                    requests to preempt precopy stream, so postcopy requests
> +#                    will be handled faster.  This is a performance feature 
> and
> +#                    should not affect the correctness of postcopy migration.
> +#                    (since 7.0)
> +#
>  # Features:
>  # @unstable: Members @x-colo and @x-ignore-shared are experimental.
>  #
> @@ -465,7 +471,7 @@
>             'block', 'return-path', 'pause-before-switchover', 'multifd',
>             'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate',
>             { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
> -           'validate-uuid', 'background-snapshot'] }
> +           'validate-uuid', 'background-snapshot', 'postcopy-preempt'] }
>  
>  ##
>  # @MigrationCapabilityStatus:
> -- 
> 2.32.0
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK