Re: [PATCH v5 02/10] ACPI ERST: specification for ERST support

[Igor Mammedov]

On Wed, 30 Jun 2021 19:26:39 +0000
Eric DeVolder <eric.devolder@oracle.com> wrote:

> Oops, at the end of the 4th paragraph, I meant to state that "Linux does not 
> support the NVRAM mode."
> rather than "non-NVRAM mode", which contradicts everything I stated prior.
> Eric.
> ________________________________
> From: Eric DeVolder <eric.devolder@oracle.com>
> Sent: Wednesday, June 30, 2021 2:07 PM
> To: qemu-devel@nongnu.org <qemu-devel@nongnu.org>
> Cc: mst@redhat.com <mst@redhat.com>; imammedo@redhat.com 
> <imammedo@redhat.com>; marcel.apfelbaum@gmail.com 
> <marcel.apfelbaum@gmail.com>; pbonzini@redhat.com <pbonzini@redhat.com>; 
> rth@twiddle.net <rth@twiddle.net>; ehabkost@redhat.com <ehabkost@redhat.com>; 
> Konrad Wilk <konrad.wilk@oracle.com>; Boris Ostrovsky 
> <boris.ostrovsky@oracle.com>
> Subject: [PATCH v5 02/10] ACPI ERST: specification for ERST support
> 
> Information on the implementation of the ACPI ERST support.
> 
> Signed-off-by: Eric DeVolder <eric.devolder@oracle.com>
> ---
>  docs/specs/acpi_erst.txt | 152 
> +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 152 insertions(+)
>  create mode 100644 docs/specs/acpi_erst.txt
> 
> diff --git a/docs/specs/acpi_erst.txt b/docs/specs/acpi_erst.txt
> new file mode 100644
> index 0000000..79f8eb9
> --- /dev/null
> +++ b/docs/specs/acpi_erst.txt
> @@ -0,0 +1,152 @@
> +ACPI ERST DEVICE
> +================
> +
> +The ACPI ERST device is utilized to support the ACPI Error Record
> +Serialization Table, ERST, functionality. The functionality is
> +designed for storing error records in persistent storage for
> +future reference/debugging.
> +
> +The ACPI specification[1], in Chapter "ACPI Platform Error Interfaces
> +(APEI)", and specifically subsection "Error Serialization", outlines
> +a method for storing error records into persistent storage.
> +
> +The format of error records is described in the UEFI specification[2],
> +in Appendix N "Common Platform Error Record".
> +
> +While the ACPI specification allows for an NVRAM "mode" (see
> +GET_ERROR_LOG_ADDRESS_RANGE_ATTRIBUTES) where non-volatile RAM is
> +directly exposed for direct access by the OS/guest, this implements
> +the non-NVRAM "mode". This non-NVRAM "mode" is what is implemented
> +by most BIOS (since flash memory requires programming operations
> +in order to update its contents). Furthermore, as of the time of this
> +writing, Linux does not support the non-NVRAM "mode".

shouldn't it be s/non-NVRAM/NVRAM/ ?

> +
> +
> +Background/Motivation
> +---------------------
> +Linux uses the persistent storage filesystem, pstore, to record
> +information (eg. dmesg tail) upon panics and shutdowns.  Pstore is
> +independent of, and runs before, kdump.  In certain scenarios (ie.
> +hosts/guests with root filesystems on NFS/iSCSI where networking
> +software and/or hardware fails), pstore may contain the only
> +information available for post-mortem debugging.

well,
it's not the only way, one can use existing pvpanic device to notify
mgmt layer about crash and mgmt layer can take appropriate measures
to for post-mortem debugging, including dumping guest state,
which is superior to anything pstore can offer as VM is still exists
and mgmt layer can inspect VMs crashed state directly or dump
necessary parts of it.

So ERST shouldn't be portrayed as the only way here but rather
as limited alternative to pvpanic in regards to post-mortem debugging
(it's the only way only on bare-metal).

It would be better to describe here other use-cases you've mentioned
in earlier reviews, that justify adding alternative to pvpanic.

> +Two common storage backends for the pstore filesystem are ACPI ERST
> +and UEFI. Most BIOS implement ACPI ERST.  UEFI is not utilized in
> +all guests. With QEMU supporting ACPI ERST, it becomes a viable
> +pstore storage backend for virtual machines (as it is now for
> +bare metal machines).
> +

> +Enabling support for ACPI ERST facilitates a consistent method to
> +capture kernel panic information in a wide range of guests: from
> +resource-constrained microvms to very large guests, and in
> +particular, in direct-boot environments (which would lack UEFI
> +run-time services).
this hunk probably not necessary

> +
> +Note that Microsoft Windows also utilizes the ACPI ERST for certain
> +crash information, if available.
a pointer to a relevant source would be helpful here.

> +Invocation
s/^^/Configuration|Usage/

> +----------
> +
> +To utilize ACPI ERST, a memory-backend-file object and acpi-erst
s/utilize/use/

> +device must be created, for example:
s/must/can/

> +
> + qemu ...
> + -object memory-backend-file,id=erstnvram,mem-path=acpi-erst.backing,
> +  size=0x10000,share=on
I'd put ^^^ on the same line as -object and use '\' at the end the 
so example could be easily copy-pasted

> + -device acpi-erst,memdev=erstnvram
> +
> +For proper operation, the ACPI ERST device needs a memory-backend-file
> +object with the following parameters:
> +
> + - id: The id of the memory-backend-file object is used to associate
> +   this memory with the acpi-erst device.
> + - size: The size of the ACPI ERST backing storage. This parameter is
> +   required.
> + - mem-path: The location of the ACPI ERST backing storage file. This
> +   parameter is also required.
> + - share: The share=on parameter is required so that updates to the
> +   ERST back store are written to the file immediately as well. Without
> +   it, updates the the backing file are unpredictable and may not
> +   properly persist (eg. if qemu should crash).

mmap manpage says:
  MAP_SHARED
             Updates to the mapping ... are carried through to the underlying 
file.
it doesn't guarantee 'written to the file immediately', though.
So I'd rephrase it to something like that:

- share: The share=on parameter is required so that updates to the ERST back 
store
         are written back to the file.

> +
> +The ACPI ERST device is a simple PCI device, and requires this one
> +parameter:
s/^.*:/and ERST device:/

> +
> + - memdev: Is the object id of the memory-backend-file.
> +
> +
> +PCI Interface
> +-------------
> +
> +The ERST device is a PCI device with two BARs, one for accessing
> +the programming registers, and the other for accessing the
> +record exchange buffer.
> +
> +BAR0 contains the programming interface consisting of just two
> +64-bit registers. The two registers are an ACTION (cmd) and a
> +VALUE (data). All ERST actions/operations/side effects happen
s/consisting of... All ERST/consisting of ACTION and VALUE 64-bit registers. 
All ERST/

> +on the write to the ACTION, by design. Thus any data needed
s/Thus//

> +by the action must be placed into VALUE prior to writing
> +ACTION. Reading the VALUE simply returns the register contents,
> +which can be updated by a previous ACTION.

> This behavior is
> +encoded in the ACPI ERST table generated by QEMU.
it's too vague, Either drop sentence or add a reference to relevant place in 
spec.


> +
> +BAR1 contains the record exchange buffer, and the size of this
> +buffer sets the maximum record size. This record exchange
> +buffer size is 8KiB.
s/^^^/
BAR1 contains the 8KiB record exchange buffer, which is the implemented maximum 
record size limit.


> +Backing File

s/^^^/Backing Storage Format/

> +------------


> +
> +The ACPI ERST persistent storage is contained within a single backing
> +file. The size and location of the backing file is specified upon
> +QEMU startup of the ACPI ERST device.

I'd drop above paragraph and describe file format here,
ultimately used backend doesn't have to be a file. For
example if user doesn't need it persist over QEMU restarts,
ram backend could be used, guest will still be able to see
it's own crash log after guest is reboot, or it could be
memfd backend passed to QEMU by mgmt layer.


> +Records are stored in the backing file in a simple fashion.
s/backing file/backend storage/
ditto for other occurrences

> +The backing file is essentially divided into fixed size
> +"slots", ERST_RECORD_SIZE in length, with each "slot"
> +storing a single record.

> No attempt at optimizing storage
> +through compression, compaction, etc is attempted.
s/^^^//

> +NOTE that any change to this value will make any pre-
> +existing backing files, not of the same ERST_RECORD_SIZE,
> +unusable to the guest.
when that can happen, can we detect it and error out?


> +Below is an example layout of the backing store file.
> +The size of the file is a multiple of ERST_RECORD_SIZE,
> +and contains N number of "slots" to store records. The
> +example below shows two records (in CPER format) in the
> +backing file, while the remaining slots are empty/
> +available.
> +
> + Slot   Record
> +        +--------------------------------------------+
> +    0   | empty/available                            |
> +        +--------------------------------------------+
> +    1   | CPER                                       |
> +        +--------------------------------------------+
> +    2   | CPER                                       |
> +        +--------------------------------------------+
> +  ...   |                                            |
> +        +--------------------------------------------+
> +    N   | empty/available                            |
> +        +--------------------------------------------+
> +        <-------------- ERST_RECORD_SIZE ------------>


> +Not all slots need to be occupied, and they need not be
> +occupied in a contiguous fashion. The ability to clear/erase
> +specific records allows for the formation of unoccupied
> +slots.
I'd drop this as not necessary


> +
> +
> +References
> +----------
> +
> +[1] "Advanced Configuration and Power Interface Specification",
> +    version 4.0, June 2009.
> +
> +[2] "Unified Extensible Firmware Interface Specification",
> +    version 2.1, October 2008.
> +
> --
> 1.8.3.1
>