Re: [Qemu-devel] [PATCH] docs/memory.txt: Clarify and expand priority/overlap documentation

[Michael S. Tsirkin]

On Sun, Sep 15, 2013 at 03:51:53PM +0100, Peter Maydell wrote:
> The documentation of how overlapping memory regions behave and how
> the priority system works was rather brief, and confusion about
> priorities seems to be quite common for developers trying to understand
> how the memory region system works, so expand and clarify it.
> This includes a worked example with overlaps, documentation of the
> behaviour when an overlapped container has "holes", and mention
> that it's valid for a region to have both MMIO callbacks and
> subregions (and how this interacts with priorities when it does).
> 
> Signed-off-by: Peter Maydell <address@hidden>

Great, thanks a lot!
Minor comments below:

> ---
>  docs/memory.txt | 48 +++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 47 insertions(+), 1 deletion(-)
> 
> diff --git a/docs/memory.txt b/docs/memory.txt
> index feb9fe9..bd0ef6e 100644
> --- a/docs/memory.txt
> +++ b/docs/memory.txt
> @@ -45,6 +45,10 @@ MemoryRegion):
>    can overlay a subregion of RAM with MMIO or ROM, or a PCI controller
>    that does not prevent card from claiming overlapping BARs.
>  
> +  It is valid for regions which are not "pure containers"

I would add "that is, MMIO, RAM or ROM"

> to have subregions;
> +  this means that any addresses within the container's region which are
> +  not claimed by a subregion

maybe stress "by any subregion"

> are handled by the container's MMIO callbacks.

RAM doesn't have MMIO callbacks (at least at the API level),
maybe say something like "cause an access to the container
itself (e.g. invoke container's MMIO callbacks or
modify container's RAM)" is better?

> +
>  - alias: a subsection of another region.  Aliases allow a region to be
>    split apart into discontiguous regions.  Examples of uses are memory banks
>    used when the guest address space is smaller than the amount of RAM
> @@ -81,6 +85,45 @@ allows the region to overlap any other region in the same 
> container, and
>  specifies a priority that allows the core to decide which of two regions at
>  the same address are visible (highest wins).
>  
> +If the higher priority region in an overlap is a container or alias, then
> +the lower priority region will appear in any "holes" that the higher priority
> +region has left by not mapping subregions
Maybe add 
"(or recursively - holes that some of the subregions
left - if some of the subregions are containers or aliases)"
> to that area of its address range.


> +For example, suppose we have a container A of size 0x8000 with two subregions
> +B and C. B is a container mapped at 0x2000, size 0x4000, priority 1; C is
> +an MMIO region mapped at 0x0, size 0x6000, priority 2. B currently has two
> +of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at
> +offset 0x2000. As a diagram:
> +
> +        0      1000   2000   3000   4000   5000   6000   7000    8000
> +        |------|------|------|------|------|------|------|-------|
> +  A:    [                                                       ]
> +  C:    [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC]
> +  B:                  [                          ]
> +  D:                  [DDDDD]
> +  E:                                [EEEEE]
> +
> +The regions that will be seen within this address range then are:
> +        [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC]
> +
> +Since B has higher priority than C, its subregions appear in the flat map
> +even where they overlap with C. In ranges where B has not mapped anything
> +C's region appears.
> +
> +If B had provided its own MMIO operations (ie it was not a pure container)
> +then these would be used for any addresses in its range not handled by
> +D or E, and the result would be:
> +        [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB]
> +
> +Priority values are local to a container, because the priorities of two
> +regions are only compared when they are both children of the same container.
> +This means that the device in charge of the container (typically modelling
> +a bus or a memory controller) can use them to manage the interaction of
> +its child regions without any side effects on other parts of the system.
> +In the example above, the priorities of D and E are unimportant because
> +they do not overlap each other. It is the relative priority of B and C
> +that causes D and E to appear on top of C: D and E's priorities are never
> +compared against the priority of C.
> +
>  Visibility
>  ----------
>  The memory core uses the following rules to select a memory region when the
> @@ -93,8 +136,11 @@ guest accesses an address:
>    - if the subregion is a leaf (RAM or MMIO), the search terminates

Maybe add
"And the leaf is selected"

>    - if the subregion is a container, the same algorithm is used within the
>      subregion (after the address is adjusted by the subregion offset)
> -  - if the subregion is an alias, the search is continues at the alias target
> +  - if the subregion is an alias, the search is continued at the alias target
>      (after the address is adjusted by the subregion offset and alias offset)
> +  - if a recursive search within a container or alias subregion does not
> +    find a match (because of a "hole" in the container's coverage of its
> +    address range), we continue with the next subregion in priority order
>  

This makes it look like the one way for a search to terminate
is with RAM or MMIO.
There are two other cases:
- non pure container -> container can be selected
- no match is found -> nothing is selected

>  Example memory map
>  ------------------
> -- 
> 1.7.11.4