Re: [PATCH V2 00/11] Live update: cpr-exec

[Steven Sistare]

On 8/5/2024 6:01 AM, Daniel P. Berrangé wrote:
> On Thu, Jul 18, 2024 at 11:56:33AM -0400, Peter Xu wrote:
> > Steve,
> >
> > On Sun, Jun 30, 2024 at 12:40:23PM -0700, Steve Sistare wrote:
> > > What?
> >
> > Thanks for trying out with the cpr-transfer series.  I saw that that series
> > missed most of the cc list here, so I'm attaching the link here:
> >
> > https://lore.kernel.org/r/1719776648-435073-1-git-send-email-steven.sistare@oracle.com
> >
> > I think most of my previous questions for exec() solution still are there,
> > I'll try to summarize them all in this reply as much as I can.
> >
> > > This patch series adds the live migration cpr-exec mode, which allows
> > > the user to update QEMU with minimal guest pause time, by preserving
> > > guest RAM in place, albeit with new virtual addresses in new QEMU, and
> > > by preserving device file descriptors.
> > >
> > > The new user-visible interfaces are:
> > >    * cpr-exec (MigMode migration parameter)
> > >    * cpr-exec-command (migration parameter)
> >
> > I really, really hope we can avoid this..
> >
> > It's super cumbersome to pass in a qemu cmdline in a qemu migration
> > parameter.. if we can do that with generic live migration ways, I hope we
> > stick with the clean approach.
>
> A further issue I have is that it presumes the QEMU configuration is
> fully captured by the command line. We have a long term design goal
> in QEMU to get away from specifying configuration on the command
> line, and move entirely to configuring QEMU via a series of QMP
> commands.
>
> This proposed command is introducing the concept of command line argv
> as a formal part of the QEMU API and IMHO that is undesirable. 

Actually cpr-exec-command does not presume anything; it is an arbitrary
command with arbitrary arguments.  If in the future QEMU takes no command-line
arguments, then mgmt will pass a simple launcher command as cpr-exec-command,
and the launcher will issue QMP commands.  Or the launcher will send a message
to another mgmt agent to do so.  It is very flexible.  Regardless, the API
definition of cpr-exec-command will not change.

As another example, in our cloud environment, when the mgmt agent starts QEMU,
it saves the QEMU args in a file.  My cpr-exec-command  is just "/bin/qemu-exec"
with a few simple arguments.  That command reads QEMU args from the file and
exec's new QEMU.

> Even
> today we have backend configuration steps only done via QMP, and I'm
> wondering how it would fit in with how mgmt apps currently doing live
> migration.

Sure, and that still works.  For live migration, mgmt starts new QEMU with
its static arguments plus -S plus -incoming, then mgmt detects QEMU has reached
the prelaunch state, then it issues QMP commands.  For live update, 
cpr-exec-command
has the static arguments plus -S, then mgmt detects QEMU has reached the 
prelaunch
state, then it issues QMP commands.

> The flipside, however, is that localhost migration via 2 separate QEMU
> processes has issues where both QEMUs want to be opening the very same
> file, and only 1 of them can ever have them open.

Indeed, and "files" includes unix domain sockets.  Network ports also conflict.
cpr-exec avoids such problems, and is one of the advantages of the method that
I forgot to promote.

- Steve