[Qemu-devel] [PATCH 21/26] qidl: add documentation

[Michael Roth]

Reviewed-by: Paolo Bonzini <address@hidden>
Signed-off-by: Michael Roth <address@hidden>
---
 docs/qidl.txt |  347 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 347 insertions(+)
 create mode 100644 docs/qidl.txt

diff --git a/docs/qidl.txt b/docs/qidl.txt
new file mode 100644
index 0000000..7d17cdc
--- /dev/null
+++ b/docs/qidl.txt
@@ -0,0 +1,347 @@
+How to Serialize Device State with QIDL
+=======================================
+
+This document describes how to implement save/restore of a device in QEMU using
+the QIDL compiler.  The QIDL compiler makes it easier to support live
+migration in devices by converging the serialization description with the
+device type declaration.  It has the following features:
+
+ 1. Single description of device state and how to serialize
+
+ 2. Fully inclusive serialization description--fields that aren't serialized
+    are explicitly marked as such including the reason why.
+
+ 3. Optimized for the common case.  Even without any special annotations,
+    many devices will Just Work out of the box.
+
+ 4. Build time schema definition.  Since QIDL runs at build time, we have full
+    access to the schema during the build which means we can fail the build if
+    the schema breaks.
+
+For the rest of the document, the following simple device will be used as an
+example.
+
+    typedef struct SerialDevice {
+        SysBusDevice parent;
+
+        uint8_t thr;            /* transmit holding register */
+        uint8_t lsr;            /* line status register */
+        uint8_t ier;            /* interrupt enable register */
+
+        int int_pending;        /* whether we have a pending queued interrupt 
*/
+        CharDriverState *chr;   /* backend */
+    } SerialDevice;
+
+Getting Started
+---------------
+
+Coverting a device struct to being serializable by QIDL, in general, only
+requires the use of the QIDL_DECLARE() macro to handle the declaration.
+The above-mentioned SerialDevice struct, for instance, can be made
+serializable simply by using the following declaration format:
+
+    typedef struct SerialDevice SerialDevice;
+
+    QIDL_DECLARE(SerialDevice) {
+        SysBusDevice parent;
+
+        uint8_t thr;            /* transmit holding register */
+        uint8_t lsr;            /* line status register */
+        uint8_t ier;            /* interrupt enable register */
+
+        int int_pending;        /* whether we have a pending queued interrupt 
*/
+        CharDriverState *chr;   /* backend */
+    };
+
+Note that the typedef is required, and must be done in advance of the actual
+struct declaration.
+
+Specifying What/How State Gets Saved
+------------------------------------
+
+By default, QIDL saves every field in a structure it sees.  This provides
+maximum correctness by default.  However, device structures generally contain
+state that reflects state that is in some way duplicated or not guest visible.
+This more often than not reflects design implementation details.
+
+Since design implementation details change over time, saving this state makes
+compatibility hard to maintain. The proper solution is to use an intermediate
+protocol to handle cross-version compatibility (for instance, a QIDL-aware
+implementation of VMState). But we can reduce churn and streamline the
+serialization/deserialization process by explicitly marking fields with
+information that QIDL can use to determine whether or not a particular field
+will be serialized. However, a serializable device implementation that fails to
+serialize state that is required to fully restore guest state is a broken one,
+so to avoid that there are very strict rules about when this is allowed, and
+what needs to be done to ensure that this does not impact correctness.
+
+There are also occasions where we want to specify *how* a field is
+serialized. Array fields for instance might rely on a size value elsewhere in
+the struct to determine the size of a dynamically-allocated array or, in the
+case of a statically-allocated array, the number of elements that have
+actually be set/initialized. There also cases where a field should only be
+serialized if another value in the struct, or even a function call, indicates
+that the field has been initialized. Markers can be used to handle these
+types of cases as well.
+
+What follows is a description of these markers/annotations and how they are
+used.
+
+## q_immutable Fields
+
+If a field is only set during device construction, based on parameters passed 
to
+the device's constructor, then there is no need to send save and restore this
+value.  We call these fields immutable and we tell QIDL about this fact by 
using
+a **q_immutable** marker.
+
+In our *SerialDevice* example, the *CharDriverState* pointer reflects the host
+backend that we use to send serial output to the user.  This is only assigned
+during device construction and never changes.  This means we can add an
+**immutable** marker to it:
+
+    QIDL_DECLARE(SerialDevice) {
+        SysBusDevice parent;
+
+        uint8_t thr;
+        uint8_t lsr;
+        uint8_t ier;
+
+        int int_pending;
+        CharDriverState q_immutable *chr;
+    };
+
+When reviewing patches that make use of the **q_immutable** marker, the 
following
+guidelines should be followed to determine if the marker is being used
+correctly.
+
+ 1. Check to see if the field is assigned anywhere other than the device
+    initialization function.
+
+ 2. Check to see if any function is being called that modifies the state of the
+    field outside of the initialization function.
+
+It can be subtle whether a field is truly immutable.  A good example is a
+*QEMUTimer*.  Timers will usually have their timeout modified with a call to
+*qemu_mod_timer()* even though they are only assigned in the device
+initialization function.
+
+If the timer is always modified with a fixed value that is not dependent on
+guest state, then the timer is immutable since it's unaffected by the state of
+the guest.
+
+On the other hand, if the timer is modified based on guest state (such as a
+guest programmed time out), then the timer carries state.  It may be necessary
+to save/restore the timer or mark it as **q_derived** and work with it
+accordingly.
+
+### q_derived Fields
+
+If a field is set based on some other field in the device's structure, then its
+value is derived.  Since this is effectively duplicate state, we can avoid
+sending it and then recompute it when we need to.  Derived state requires a bit
+more handling than immutable state.
+
+In our *SerialDevice* example, our *int_pending* flag is really derived from
+two pieces of state.  It is set based on whether interrupts are enabled in the
+*ier* register and whether the *THRE* flag is not set in the *lsr*
+register.
+
+To mark a field as derived, use the **derived** marker.  To update our
+example, we would do:
+
+    QIDL_DECLARE(SerialDevice) {
+        SysBusDevice parent;
+
+        uint8_t thr;
+        uint8_t lsr;
+        uint8_t ier;
+
+        int q_derived int_pending;
+        CharDriverState q_immutable *chr;
+    };
+
+There is one other critical step needed when marking a field as derived.  A
+*post_load* function must be added that updates this field after loading the
+rest of the device state.  This function is implemented in the device's source
+file, not in the QIDL header.  Below is an example of what this function may 
do:
+
+    static void serial_post_load(SerialDevice *s)
+    {
+        s->int_pending = !(s->lsr & THRE) && (s->ier & INTE);
+    }
+
+When reviewing a patch that marks a field as *derived*, the following criteria
+should be used:
+
+ 1. Does the device have a post load function?
+
+ 2. Does the post load function assign a value to all of the derived fields?
+
+ 3. Are there any obvious places where a derived field is holding unique state?
+
+### q_broken Fields
+
+QEMU does migration with a lot of devices today.  When applying this 
methodology
+to these devices, one will quickly discover that there are a lot of fields that
+are not being saved today that are not derived or immutable state.
+
+These are all bugs.  It just so happens that these bugs are usually not very
+serious.  In many cases, they cause small functionality glitches that so far
+have not created any problems.
+
+Consider our *SerialDevice* example.  In QEMU's real *SerialState* device, the
+*thr* register is not saved, yet we have not marked it immutable or derived.
+
+The *thr* register is a temporary holding register that the next character to
+transmit is placed in while we wait for the next baud cycle.  In QEMU, we
+emulate a very fast baud rate regardless of what guest programs.  This means
+that the contents of the *thr* register only matter for a very small period of
+time (measured in microseconds).
+
+The likelihood of a migration converging in that very small period of time when
+the *thr* register has a meaningful value is very small.  Moreover, the worst
+thing that can happen by not saving this register is that we lose a byte in the
+data stream.  Even if this has happened in practice, the chances of someone
+noticing this as a bug is pretty small.
+
+Nonetheless, this is a bug and needs to be eventually fixed.  However, it would
+be very inconvenient to constantly break migration by fixing all of these bugs
+one-by-one.  Instead, QIDL has a **broken** marker.  This indicates that a 
field
+is not currently saved, but should be in the future.
+
+In general, q_broken markers should never be introduced in new code, and should
+be used instead as a development aid to avoid serialization issues while
+writing new device code.
+
+Below is an update of our example to reflect our real life serial device:
+
+    QIDL_DECLARE(SerialDevice) {
+        SysBusDevice parent;
+
+        uint8_t q_broken thr;
+        uint8_t lsr;
+        uint8_t ier;
+
+        int q_derived int_pending;
+        CharDriverState q_immutable *chr;
+    };
+
+When reviewing the use of the broken marker, the following things should be
+considered:
+
+ 1. What are the ramifications of not sending this data field?
+
+ 2. If not sending this data field can cause data corruption or very poor
+    behavior within the guest, the broken marker is not appropriate to use.
+
+ 3. Assigning a default value to a field can also be used to fix a broken field
+    without significantly impacting live migration compatibility.
+
+### q_elsewhere fields
+
+In some cases state is saved-off when serializing a separate device
+structure. For example, IDEState stores a reference to an IDEBus structure:
+
+    QIDL_DECLARE(IDEState) {
+        IDEBus q_elsewhere *bus;
+        uint8_t unit;
+        ...
+    };
+
+However, IDEState is actually a member of IDEBus, so would have already been
+serialized in the process of serializing IDEBus:
+
+    QIDL_DECLARE(IDEBus) {
+        BusState qbus;
+        IDEDevice *master;
+        IDEDevice *slave;
+        IDEState ifs[2];
+        ...
+    };
+
+To handle this case we've used the *q_elsewhere* marker to note that the
+IDEBus* field in IDEState should not be saved since that is handled
+elsewhere.
+
+### q_optional fields
+
+Some state is only serialized in certain circumstances. To handle these cases
+you can specify a ***q_optional*** marker, which will, for a particular field
+"fieldname", tell QIDL to reference the field "has_fieldname" (of type bool)
+in the same struct to determine whether or not to serialize "fieldname". For
+example, if the data field was optionally serialized, you could do following:
+
+    QIDL_DECLARE(SerialFIFO) {
+        bool has_data;
+        uint8_t q_optional data[UART_FIFO_LENGTH];
+        uint8_t count;
+        uint8_t itl;
+        uint8_t tail;
+        uint8_t head;
+    };
+
+Of course, your device code will need to be updated to set has_data when
+appropriate. If has_data is set based on guest state, then it must be
+serialized as well.
+
+Arrays
+------
+
+QIDL has support for multiple types of arrays.  The following sections describe
+the different rules for arrays.
+
+Fixed Sized Arrays
+------------------
+
+A fixed sized array has a size that is known at build time.  A typical example
+would be:
+
+    QIDL_DECLARE(SerialFIFO) {
+        uint8_t data[UART_FIFO_LENGTH];
+        uint8_t count;
+        uint8_t itl;
+        uint8_t tail;
+        uint8_t head;
+    };
+
+In this example, *data* is a fixed sized array.  No special annotation is 
needed
+for QIDL to marshal this area correctly.  The following guidelines apply to
+fixed sized arrays:
+
+ 1. The size of the array is part of the device ABI.  It should not change
+    without renaming the field.
+
+Variable Sized, Fixed Capacity Arrays
+-------------------------------------
+
+Sometimes it's desirable to have a variable sized array.  QIDL currently
+supports variable sized arrays provided that the maximum capacity is fixed and
+part of the device structure memory.
+
+A typical example would be a slightly modified version of our above example:
+
+    QIDL_DECLARE(SerialFIFO) {
+        uint8_t count;
+        uint8_t q_size(count) data[UART_FIFO_LENGTH];
+        uint8_t itl;
+        uint8_t tail;
+        uint8_t head;
+    };
+
+In this example, *data* is a variable sized array with a fixed capacity of
+*UART_FIFO_LENGTH*.  When we serialize, we want only want to serialize *count*
+members.
+
+The ABI implications of capacity are a bit more relaxed with variable sized
+arrays.  In general, you can increase or decrease the capacity without breaking
+the ABI although you may cause some instances of migration to fail between
+versions of QEMU with different capacities.
+
+When reviewing variable sized, fixed capacity arrays, keep the following things
+in mind:
+
+ 1. The variable size must occur before the array element in the state
+    structure.
+
+ 2. The capacity can change without breaking the ABI, but care should be used
+    when making these types of changes.
-- 
1.7.9.5