[Qemu-devel] [PATCH] hmp: Improve 'info mtree' with optional parm for mapinfo

[Thorsten Kohfeldt]

From: Thorsten Kohfeldt <address@hidden>
Date: Wed, 31 Aug 2016 22:43:14 +0200
Subject: [PATCH] hmp: Improve 'info mtree' with optional parm for mapinfo

Motivation
When 'tuning' 'quirks' for VFIO imported devices, it is not easy to
directly grasp the implications of the priorisation algorithms in place
for the 'layered mapping' of memory regions.
Even though there are rules (documented in docs/memory.txt), once in a
while one might question the correctness of the actual implementation of
the rules.
Particularly, I believe I have uncovered a divergence of (sub-)region
priorisation/order/visibility in a corner case of importing a device
(which requires a 'quirk') with mmap enabled vs. mmap disabled.
This modification provides a means of visualising the ACTUAL
mapping/visibility/occlusion of subregions within regions, whereas the
current info mtree command only lists the tree of regions (all, visible
and invisible ones).
It is primarily intended to provide support for easy presentation of my
cause, but I strongly believe this modification also has general purpose
advantages.

Functional implementation
A new OPTIONAL integer parameter, mapinfo-width, is added to the
monitor/hmp 'info mtree' command.
Effect:
When given and between 5 and 257, then each mtree output line is
prefixed with <mapinfo-width> columns of 'visibility samples', depicting
what qemu's memory region priorisation algorithms effectively make
visible/accessible at that sample address at the time of inquiry.
NOTE that
the sampling algorithm virtually cuts the memory region into (width - 1)
'slices' and computes (width) samples at the edges of those virtual
slices. Thus, it is probably a good idea to always request (2^n + 1)
samples.
ALSO NOTE that
the memory regions are NOT actually accessed at those 'samples', ONLY a
region PRIORISATION EVALUATION is performed for the sample addresses.
You can setup a default using environment variable
QEMU_INFO_MTREE_MAPINFO_WIDTH (must be in the Qemu instance's
environment; when unset, then the default is 0, i.e. off).
Giving negative values for sample-width results in using that default,
while values above 257 are reduced to 257, and values from 0 to 4 switch
the sampling off.

Technical implementation
3 functions are added to memory.c:
sane_mtree_info_sample_width() is used to sanitise the new parameter,
i.e. to provide a default and adjust it towards 'usability'. It is
called by existing function mtree_info().
mtree_print_mr_v_samples() is called for each mtree memory region (mr)
output line in order to print the 'mapinfo' prefix. The call is
performed by existing function mtree_print_mr() with parameter
'const MemoryRegion *mr', thus promising the object under investigation
is not modified.
mtree_mr_sample_reftype_marker() is used to traverse an mr subtree for a
given hardware address in order to basically find the first matching
enabled region and return its type. It is called by
mtree_print_mr_v_samples() for <sanitised mapinfo-width> 'sample'
addresses. As an mr tree is traversed, limited recursion is involved.
Additionally, for existing functions memory_region_to_address_space(),
memory_region_size(), and memory_region_find_rcu() their parameter
'MemoryRegion *mr' had to be constrained to 'const MemoryRegion *mr' in
order to satisfy the compiler while attempting to emphasize the
'object is not modified' promise by insisting to pass *mr as const into
mtree_print_mr_v_samples(). This did not entail changes in the bodies of
mentioned functions.

Signed-off-by: Thorsten Kohfeldt <address@hidden>
---
 hmp-commands-info.hx  |   9 +-
 include/exec/memory.h |   7 +-
 memory.c              | 246 ++++++++++++++++++++++++++++++++++++++++++++++++--
 monitor.c             |   4 +-
 4 files changed, 250 insertions(+), 16 deletions(-)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 74446c6..1593238 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -264,16 +264,17 @@ ETEXI

     {
         .name       = "mtree",
-        .args_type  = "",
-        .params     = "",
-        .help       = "show memory tree",
+        .args_type  = "mapinfo-width:l?",
+        .params     = "[mapinfo-width]",
+        .help       = "show memory tree "
+                      "(mapinfo-width: depict memory subregion mappings with 
leading characters)",
         .mhandler.cmd = hmp_info_mtree,
     },

 STEXI
 @item info mtree
 @findex mtree
-Show memory tree.
+Show memory tree optionally depicting subregion mappings.
 ETEXI

     {
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 3e4d416..751483c 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -536,7 +536,7 @@ struct Object *memory_region_owner(MemoryRegion *mr);
  *
  * @mr: the memory region being queried.
  */
-uint64_t memory_region_size(MemoryRegion *mr);
+uint64_t memory_region_size(const MemoryRegion *mr);

 /**
  * memory_region_is_ram: check whether a memory region is random access
@@ -1202,7 +1202,10 @@ void memory_global_dirty_log_start(void);
  */
 void memory_global_dirty_log_stop(void);

-void mtree_info(fprintf_function mon_printf, void *f);
+/**
+ * mtree_info: see hmp-commands-info.hx item info mtree
+ */
+void mtree_info(fprintf_function mon_printf, void *f, const int mapinfo_width);

 /**
  * memory_region_dispatch_read: perform a read directly to the specified
diff --git a/memory.c b/memory.c
index 0eb6895..99161bd 100644
--- a/memory.c
+++ b/memory.c
@@ -595,7 +595,7 @@ static MemTxResult access_with_adjusted_size(hwaddr addr,
     return r;
 }

-static AddressSpace *memory_region_to_address_space(MemoryRegion *mr)
+static AddressSpace *memory_region_to_address_space(const MemoryRegion *mr)
 {
     AddressSpace *as;

@@ -1477,7 +1477,7 @@ void memory_region_unref(MemoryRegion *mr)
     }
 }

-uint64_t memory_region_size(MemoryRegion *mr)
+uint64_t memory_region_size(const MemoryRegion *mr)
 {
     if (int128_eq(mr->size, int128_2_64())) {
         return UINT64_MAX;
@@ -2062,11 +2062,11 @@ bool memory_region_is_mapped(MemoryRegion *mr)
 /* Same as memory_region_find, but it does not add a reference to the
  * returned region.  It must be called from an RCU critical section.
  */
-static MemoryRegionSection memory_region_find_rcu(MemoryRegion *mr,
+static MemoryRegionSection memory_region_find_rcu(const MemoryRegion *mr,
                                                   hwaddr addr, uint64_t size)
 {
     MemoryRegionSection ret = { .mr = NULL };
-    MemoryRegion *root;
+    const MemoryRegion *root;
     AddressSpace *as;
     AddrRange range;
     FlatView *view;
@@ -2324,10 +2324,172 @@ struct MemoryRegionList {

 typedef QTAILQ_HEAD(queue, MemoryRegionList) MemoryRegionListHead;

+static char mtree_mr_sample_reftype_marker(const MemoryRegion *mr,
+                                           const MemoryRegion *mv,
+                                           const MemoryRegion *mc,
+                                           const hwaddr offset,
+                                           const int max_chainlen)
+{
+    const MemoryRegion *al = mc->alias;
+    const MemoryRegion *submr;
+    char marker, hint = 0;
+
+    if (int128_ge(int128_make64(offset), mc->size)) {
+        return 0;
+    }
+    if (max_chainlen < 0) {
+        /* this is most probably a complex alias loop situation */
+        return 'E'; /* max. link chain length exceeded */
+    }
+
+    if (al) {
+        if (al == mv) {
+            if (mr->enabled) {
+                if (mc == mr) {
+                    return '@'; /* indirect link found */
+                } else {
+                    return 'a'; /* 2nd degree related alias */
+                }
+            } else {
+                return '~';
+            }
+        }
+        if (al == mr) {
+            return 'L'; /* alias loop */
+        }
+        if (al == mc) {
+            return 'X'; /* alias self reference */
+        }
+        if (al->alias == mc) {
+            return 'M'; /* mutual alias */
+        }
+        if ((al->enabled) &&
+            (int128_lt(int128_add(int128_make64(offset),
+                                  int128_make64(mc->alias_offset)),
+                       al->size))) {
+            marker = mtree_mr_sample_reftype_marker(mr, mv, al,
+                                                    offset + mc->alias_offset,
+                                                    max_chainlen - 1);
+            if (marker && marker != 'E') {
+                return marker;
+            }
+            hint |= marker; /* propagate 'E' dominantly over 0 */
+        }
+    }
+
+    QTAILQ_FOREACH(submr, &mc->subregions, subregions_link) {
+        if (submr == mv) {
+            if (mr->enabled) {
+                if (mc == mr) {
+                    return 's'; /* occluded by subregion */
+                } else {
+                    return 'c'; /* 2nd degree related child */
+                }
+            } else {
+                return '.';
+            }
+        }
+        if ((submr->enabled) && (offset >= submr->addr)) {
+            marker = mtree_mr_sample_reftype_marker(mr, mv, submr,
+                                                    offset - submr->addr,
+                                                    max_chainlen - 1);
+            if (marker && marker != 'E') {
+                return marker;
+            }
+            hint |= marker; /* propagate 'E' dominantly over 0 */
+        }
+    }
+
+    return hint; /* either 0 or 'E' */
+}
+
+/* Depict memory region subrange structure,
+ * i.e. occlusion by submaps respectively visibility of submaps
+ * as supposedly resulting from region priorisation rules.
+ */
+static void mtree_print_mr_v_samples(fprintf_function mon_printf,
+                                     void *f,
+                                     const MemoryRegion *mr,
+                                     const unsigned int columns)
+{
+    const MemoryRegion *mv;
+    unsigned int i, j;
+    hwaddr size, offset;
+    bool covered = false;
+
+    /* prevent uncovered corner cases and excessive sampling effort */
+    if ((columns < 2) || (columns > 1000)) {
+        if (columns) {
+            mon_printf(f, "[%s: not supporting %d column(s)]",
+                       __func__, columns);
+        }
+        return;
+    }
+
+    /* convert/constrain mr->size to hwaddr bit size */
+    size = memory_region_size(mr);
+    if (!size) {
+        /* size is 0 for example with 'dummy' regions in VFIO */
+        mon_printf(f, "%*s", columns, "");
+        return;
+    }
+
+    i = columns;
+    j = size / (i - 1); /* step ('slice') width */
+    if (j < 1) {
+        j = 1;
+    }
+    offset = 0; /* sample position within region */
+    while (i--) {
+        if (offset >= (size - 1)) {
+            /* region might have less bytes than columns were requested */
+            if (covered) {
+                mon_printf(f, " "); /* no more additional samples */
+                continue;
+            }
+            covered = true;
+            offset = size - 1;
+        }
+        rcu_read_lock();
+        mv = memory_region_find_rcu(mr, offset, 1).mr;
+        rcu_read_unlock();
+        if (mv == mr) {
+            if (mr->enabled) {
+                mon_printf(f, "+"); /* mr is directly visible at the sample 
addr */
+            } else {
+                mon_printf(f, "-");
+            }
+        } else {
+            /* mr is not visible, but mv is visible unless NULL */
+            if (!mv) {
+                mon_printf(f, "/"); /* nothing mapped at the sample addr */
+            } else {
+                /* mr is occluded by mv which supposedly is
+                 * either linked via an alias/subregion construct
+                 *     or a higher prioritized subregion of the same parent
+                 */
+                char marker;
+
+                /* current memory region hierarchy depth is close to 5,
+                 * so a maximum search depth of 20 should be sufficient;
+                 * i.e. a link chain length longer than 20 is considered a loop
+                 */
+                marker = mtree_mr_sample_reftype_marker(mr, mv, mr, offset, 
20);
+                if (!marker) {
+                    marker = 'o'; /* occlusion by sibling or unrelated region 
*/
+                }
+                mon_printf(f, "%c", marker);
+            }
+        }
+        offset += j;
+    }
+}
+
 static void mtree_print_mr(fprintf_function mon_printf, void *f,
                            const MemoryRegion *mr, unsigned int level,
                            hwaddr base,
-                           MemoryRegionListHead *alias_print_queue)
+                           MemoryRegionListHead *alias_print_queue,
+                           const unsigned int mr_visibility_samples)
 {
     MemoryRegionList *new_ml, *ml, *next_ml;
     MemoryRegionListHead submr_print_queue;
@@ -2338,6 +2500,12 @@ static void mtree_print_mr(fprintf_function mon_printf, 
void *f,
         return;
     }

+    if (mr_visibility_samples) {
+        /* on the very left depict region occlusion/visibility */
+        mtree_print_mr_v_samples(mon_printf, f,
+                                 mr, mr_visibility_samples);
+    }
+
     for (i = 0; i < level; i++) {
         mon_printf(f, "  ");
     }
@@ -2415,7 +2583,7 @@ static void mtree_print_mr(fprintf_function mon_printf, 
void *f,

     QTAILQ_FOREACH(ml, &submr_print_queue, queue) {
         mtree_print_mr(mon_printf, f, ml->mr, level + 1, base + mr->addr,
-                       alias_print_queue);
+                       alias_print_queue, mr_visibility_samples);
     }

     QTAILQ_FOREACH_SAFE(ml, &submr_print_queue, queue, next_ml) {
@@ -2423,24 +2591,84 @@ static void mtree_print_mr(fprintf_function mon_printf, 
void *f,
     }
 }

-void mtree_info(fprintf_function mon_printf, void *f)
+static int sane_mtree_info_mapinfo_width(const int requested_width)
+{
+    static int default_width = -1;
+
+    int width = requested_width;
+
+    /* (on first call) establish a default for the number of region samples */
+    if (default_width < 0) {
+        char *str = getenv("QEMU_INFO_MTREE_MAPINFO_WIDTH");
+
+        if (str) {
+            default_width = atoi(str);
+        } else {
+            default_width = 0;
+        }
+        if (default_width < 0) {
+            /* prevent repeating getenv - fallback to sampling over 8 'slices' 
*/
+            default_width = 9;
+        }
+    }
+
+    /* use the default when the requested width is negative */
+    if (width < 0) {
+        width = default_width;
+    }
+
+    /* and finally adapt to 'usability' constraints */
+    if (width < 5) {
+        /* sampling over very few 'slices' creates too much
+         * corner case complexity and is also not much of use
+         */
+        width = 0; /* disable the sample display completely */
+    }
+    if (width > 0x101) {
+        /* limit the output prefix width to some (un-)reasonable value:
+         * max 2^8+1 samples subdividing a region into 2^8 'slices'
+         */
+        width = 0x101;
+    }
+
+    return width;
+}
+
+void mtree_info(fprintf_function mon_printf, void *f, const int mapinfo_width)
 {
     MemoryRegionListHead ml_head;
     MemoryRegionList *ml, *ml2;
     AddressSpace *as;
+    int mr_visibility_samples = sane_mtree_info_mapinfo_width(mapinfo_width);
+
+    if (mr_visibility_samples) {
+        /* print a symbolisation cross reference */
+        mon_printf(f, "\n");
+        mon_printf(f, "/: nothing mapped at sample address\n");
+        mon_printf(f, "+: region directly mapped at sample\n");
+        mon_printf(f, "@: alias region mapped at sample\n");
+        mon_printf(f, "~: alias region mappable but disabled at sample\n");
+        mon_printf(f, "s: region occluded by subregion at sample\n");
+        mon_printf(f, "a: region occluded by an aliased subregion at 
sample\n");
+        mon_printf(f, "c: region occluded by a child (subregion) of an alias at 
sample\n");
+        mon_printf(f, "o: region occluded by some other region at sample\n");
+        mon_printf(f, "\n");
+    }

     QTAILQ_INIT(&ml_head);

     QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {
         mon_printf(f, "address-space: %s\n", as->name);
-        mtree_print_mr(mon_printf, f, as->root, 1, 0, &ml_head);
+        mtree_print_mr(mon_printf, f, as->root, 1, 0, &ml_head,
+                       mr_visibility_samples);
         mon_printf(f, "\n");
     }

     /* print aliased regions */
     QTAILQ_FOREACH(ml, &ml_head, queue) {
         mon_printf(f, "memory-region: %s\n", memory_region_name(ml->mr));
-        mtree_print_mr(mon_printf, f, ml->mr, 1, 0, &ml_head);
+        mtree_print_mr(mon_printf, f, ml->mr, 1, 0, &ml_head,
+                       mr_visibility_samples);
         mon_printf(f, "\n");
     }

diff --git a/monitor.c b/monitor.c
index 5c00373..2f55d12 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1527,7 +1527,9 @@ static void hmp_boot_set(Monitor *mon, const QDict *qdict)

 static void hmp_info_mtree(Monitor *mon, const QDict *qdict)
 {
-    mtree_info((fprintf_function)monitor_printf, mon);
+    int mapinfo_width = qdict_get_try_int(qdict, "mapinfo-width", -1);
+
+    mtree_info((fprintf_function)monitor_printf, mon, mapinfo_width);
 }

 static void hmp_info_numa(Monitor *mon, const QDict *qdict)
--
2.5.5