[Qemu-devel] [PATCH v4 0/4] Migration Deciphering aid

[Alexander Graf]

Migration is a black hole to most people. One of the biggest reasons for
this is that its protocol is a secret, undocumented sauce of code rolling
around random parts of the QEMU code base.

But what if we simply exposed the description of how the format looks like
alongside the actual migration stream? This is what this patch set does.

It adds a new section that comes after the end of stream marker (so that it
doesn't slow down migration) that contains a JSON description of the device
state description.

Along with this patch set also comes a python script that can read said JSON
from a migration dump and decipher the device state and ram contents of the
migration dump using it.

With this, you can now fully examine all glorious details that go over the
wire when virtual machine state gets dumped, such as during live migration.

We discussed the approach taken here during KVM Forum 2013. Originally, my idea
was to include a special device that contains the JSON data which can be enabled
on demand. Anthony suggested however to just always include the description data
after the end marker which I think is a great idea.

  Example decoded migration: http://csgraf.de/mig/mig.txt
  Example migration description: http://csgraf.de/mig/mig.desc.txt
  Presentation: https://www.youtube.com/watch?v=iq1x40Qsrew
  Slides: https://www.dropbox.com/s/otp2pk2n3g087zp/Live%20Migration.pdf

v1 -> v2:

  - a lot, v1 was from 2013

v2 -> v3:

  - QOMify the QJSON object, makes for easier destruction
  - improve ftell_fast, now works with bdrv too
  - Dont compress objects with subsections
  - Destroy QJSON object
  - Add tell function to MigrationFile
  - Report where subsections were not found
  - Print ram sections with size
  - Remove foreign desc file support
  - Add memory dump support (-m option)
  - Add -x option to extract all sections into files

v3 -> v4:

  - Squash properly
  - Remove unused field from struct
  - Update patch description
  - Change copyright date to 2015

Alexander Graf (4):
  QJSON: Add JSON writer
  qemu-file: Add fast ftell code path
  migration: Append JSON description of migration stream
  Add migration stream analyzation script

 Makefile.objs                 |   1 +
 hw/pci/pci.c                  |   2 +-
 hw/scsi/spapr_vscsi.c         |   2 +-
 hw/virtio/virtio.c            |   2 +-
 include/migration/migration.h |   1 +
 include/migration/qemu-file.h |   1 +
 include/migration/vmstate.h   |   3 +-
 include/qjson.h               |  29 +++
 migration/qemu-file.c         |  16 ++
 migration/vmstate.c           | 186 ++++++++++++-
 qjson.c                       | 129 +++++++++
 savevm.c                      |  54 +++-
 scripts/analyze-migration.py  | 592 ++++++++++++++++++++++++++++++++++++++++++
 tests/test-vmstate.c          |   6 +-
 14 files changed, 1005 insertions(+), 19 deletions(-)
 create mode 100644 include/qjson.h
 create mode 100644 qjson.c
 create mode 100755 scripts/analyze-migration.py

-- 
1.7.12.4