[COMMITTED] poke-elf.texi: more documentation for the ELF pickles

[Jose E. Marchesi]

---
 poke-elf.texi | 499 +++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 373 insertions(+), 126 deletions(-)

diff --git a/poke-elf.texi b/poke-elf.texi
index ef25dfc..722089a 100644
--- a/poke-elf.texi
+++ b/poke-elf.texi
@@ -57,11 +57,17 @@ Getting Started
 Poking at ELF
 * ELF Configurations::            The ELF Configuration Registry.
 * ELF Basic Types::               Simple types.
-* ELF Files::                     ELF file, ELF header and file tables.
+* ELF File::                      32-bit and 64-bit ELF files.
+* ELF Header::                    The ELF file header.
+* ELF Section Headers::           Entry in the ELF section header table.
+* ELF Program Headers::           Entry in the ELF program header table.
 @c * ELF Sections::                  ELF sections and their contents.
 @c * ELF Segments::                  ELF segments and their contents.
-@c * ELF Symbols::                   ELF symbols and their attributes.
-@c * ELF Machines::                  ELF machine specific data structures.
+* ELF Symbols::                   Symbols and their attributes.
+* ELF Notes::                     Auxiliary information.
+* ELF Relocations::               REL and RELA relocations.
+* ELF Dynamic Info::              Stored information for dynamic linking.
+* ELF Machines::                  ELF machine specific data structures.
 
 Extending and Hacking
 @c * Adding a new ELF machine
@@ -406,9 +412,9 @@ values this parameter may adopt.
 
 The @code{class} argument is a string that gives a name to the new
 configuration parameter.  These have names like
-@code{"relocation-types"} and @code{"file-classes"}.  Our ELF pickles
-use a definite set of names, documented below, but nothing prevents
-you to use your own.
+@code{relocation-types} or @code{file-classes}.  Our ELF pickles use a
+definite set of names, documented below, but nothing prevents you to
+use your own.
 
 This is how we would add a couple of common relocation types to the
 register (note the actual ELF specification has none of these, they
@@ -654,13 +660,14 @@ relocation type in RISCV:
 0
 @end example
 
-Turns out it isn't :D
+Turns out it doesn't! :D
 
 @section Using configuration parameters in types
 
 The formatting and checking methods described above are mainly used in
-this pickle in order to implement pretty-printers and data integrity
-constraints in the several ELF structures holding such values.
+the ELF pickles in order to implement pretty-printers and data
+integrity constraints in the several ELF structures holding such
+values.
 
 @noindent
 For example:
@@ -687,7 +694,7 @@ type Elf64_Shdr =
 @end example
 
 However, they are also very useful to the user while poking at
-existing data (``if these bytes were to be interpreted as section
+existing data (``if these bytes were to be interpreted as ELF section
 flags in some given arch, which ones they would be?''), composing new
 data and also when generating reports and statistics.
 
@@ -744,24 +751,16 @@ An ELF @dfn{unsigned program address}.
 And ELF @dfn{unsigned file offset}.
 @end table
 
-@node ELF Files
-@chapter ELF Files
-
-@menu
-* ELF File::                      32-bit and 64-bit ELF files.
-* ELF Header::                    The ELF file header.
-* ELF Section Header Table::      The ELF file sections.
-* ELF Program Header Table::      The ELF file segments.
-@end menu
-
 @node ELF File
-@section ELF File
+@chapter ELF File
 @cindex @code{Elf32_File}
 @cindex @code{Elf64_File}
 
 The Poke types provided to denote ELF64 and ELF32 files are
 @code{Elf64_File} and @code{Elf32_File} respectively.
 
+@section Overview
+
 @example
 type Elf32_File =
   struct
@@ -790,8 +789,7 @@ type Elf64_File =
   @};
 @end example
 
-@noindent
-Where:
+@section Fields
 
 @table @code
 @item ehdr
@@ -816,58 +814,23 @@ This table, if it exists, can be located anywhere in the 
ELF file.
 The ELF header determines the size and location of the table.
 @end table
 
-@noindent
-Poking at an ELF file usually starts by opening some IO space and
-mapping a @code{Elf64_File} (or @code{Elf32_File}):
-
-@example
-(poke) .file /bin/ls
-(poke) var elf = Elf64_File @@ 0#B
-@end example
-
-@noindent
-Once mapped, we can access any of the above fields.  For example,
-let's see how many sections and segments this file has:
-
-@example
-(poke) elf.shdr'length
-30UL
-(poke) elf.phdr'length
-11UL
-@end example
-
-@noindent
-In case the file didn't have a program header table, which always
-happens with object files, we would have got an exception if we tried
-to access the absent field @code{phdr}:
-
-@example
-$ echo '' | gcc -c -xc -o foo.o -
-$ poke foo.o
-(poke) load elf
-(poke) (Elf64_File @@ 0#B).phdr
-unhandled invalid element exception
-@end example
-
-Both @code{Elf64_File} and @code{Elf32_File} provide several handy
-methods that are documented in the subsections below.
+@section Methods
 
 @subsection Methods related to sections
 
-Unlike in older object formats (like a.out for example) the sections
-present in ELF files are not fixed nor they have fixed pre-defined
-names: there can be any number of them (including none) and they can
-have any arbitrary name.  Also, more than one section in the file can
-have the same name.
+@table @code
+@item File_Elf64.section_name_p = (string @var{name}) int<32>
+@itemx File_Elf32.section_name_p = (string @var{name}) int<32>
 
-So when it comes to ELF files, the process to determine whether one or
-more section with a given name exists in the file is a bit laborious:
-one has to traverse the section header table, fetch the section names
-from whatever appropriate string table, etc.
+Given a section @var{name}, return whether a section with that name
+exists in the ELF file.
 
-The following methods, that you can use in your own pickles, scripts,
-or at the prompt, are handy to look at particular sections in the
-file.
+@item File_Elf64.get_sections_by_name = (string @var{name}) Elf64_Shdr[]
+@itemx File_Elf32.get_sections_by_name = (string @var{name}) Elf32_Shdr[]
+
+Given the @var{name} of a section, return an array of section headers
+in the ELF file having that name.  The returned array may of course be
+empty.
 
 For example, this is how you can get an array of all the sections in
 the file with name @code{.text}:
@@ -888,21 +851,8 @@ the file with name @code{.text}:
 @}]
 @end example
 
-@table @code
-@item section_name_p = (string @var{name}) int<32>
-
-Given a section @var{name}, return whether a section with that name
-exists in the ELF file.
-
-@item get_sections_by_name = (string @var{name}) Elf64_Shdr[]
-@itemx get_sections_by_name = (string @var{name}) Elf32_Shdr[]
-
-Given the @var{name} of a section, return an array of section headers
-in the ELF file having that name.  The returned array may of course be
-empty.
-
-@item get_sections_by_type = (Elf_Word @var{stype}) Elf64_Shdr[]
-@itemx get_sections_by_type = (Elf_Word @var{stype}) Elf32_Shdr[]
+@item File_Elf64.get_sections_by_type = (Elf_Word @var{stype}) Elf64_Shdr[]
+@itemx File_Elf32.get_sections_by_type = (Elf_Word @var{stype}) Elf32_Shdr[]
 
 Given a section type (one of the @code{ELF_SHT_*} value) return an
 array of section headers in the ELF file with that type.  The returned
@@ -911,18 +861,9 @@ array may be empty.
 
 @subsection Methods related to string tables
 
-The names of several entities in ELF files are stored in different
-string table, which are themselves stored in different sections.
-There are different rules establishing where exactly the name of
-certain entities (sections, symbols, ...) are to be found.
-
-These rules are not trivial and require traversing several data
-structures.  Therefore the @code{Elf64_File} (and @code{File32_File})
-type provides several methods in order to easily determine the name of
-these entities.
-
 @table @code
-@item get_section_name = (offset<Elf_Word,B> @var{offset}) string
+@item Elf64_File.get_section_name = (offset<Elf_Word,B> @var{offset}) string
+@itemx Elf32_File.get_section_name = (offset<Elf_Word,B> @var{offset}) string
 
 Given an offset into the ELF file's section string table, return the
 string starting at that @var{offset}.  This uses one particular string
@@ -938,14 +879,15 @@ in an ELF file@footnote{The first section in an ELF file 
is the
 ".interp"
 @end example
 
-@item get_symbol_name = (Elf64_Shdr @var{symtab}, offset<Elf_Word,B> 
@var{offset}) string
-@itemx get_symbol_name = (Elf32_Shdr @var{symtab}, offset<Elf_Word,B> 
@var{offset}) string
+@item Elf64_File.get_symbol_name = (Elf64_Shdr @var{symtab}, 
offset<Elf_Word,B> @var{offset}) string
+@itemx Elf64_File.get_symbol_name = (Elf32_Shdr @var{symtab}, 
offset<Elf_Word,B> @var{offset}) string
 
 Given the section header of a section that contains a symbol table
 @var{symtab}, and an @var{offset}, return the corresponding string
 stored at the symbol table associated string table.
 
-@item get_string = (offset<Elf_Word,B> offset) string
+@item Elf64_File.get_string = (offset<Elf_Word,B> offset) string
+@itemx Elf32_File.get_string = (offset<Elf_Word,B> offset) string
 
 Given an @var{offset}, return the string stored at that offset in the
 ``default'' string table of the ELF file.
@@ -954,40 +896,27 @@ The default string table is contained in a section named
 @code{.strtab}.  If such a section doesn't exist, or if it exists but
 it doesn't contain a string table, then this function raises
 @code{E_inval}.
-
 @end table
 
 @subsection Methods related to section groups
 
-ELF supports grouping several sections in a @dfn{section group}.  This
-is useful when several sections have to go together, because they rely
-on each other somehow.
-
-A section of type @code{SHT_GROUP} defines a section group.  Groups
-are univocally identified by a @dfn{group signature}, which is the
-name associated with a symbol that is stored in a particular symbol
-table, linked from the section header of the group defining section.
-
-Again, it is not exactly trivial to determine, for example, which of
-the sections in the ELF file pertain to which group.  Therefore the
-pickle provides the methods below:
-
 @table @code
-@item get_group_signature = (Elf64_Shdr @var{section}) string
-@itemx get_group_signature = (Elf32_Shdr @var{section}) string
+@item Elf64_File.get_group_signature = (Elf64_Shdr @var{section}) string
+@itemx Elf32_File.get_group_signature = (Elf32_Shdr @var{section}) string
 
 Return the signature corresponding to a given group @var{section},
 characterized by its entry in the section header table.  If the given
 section header doesn't correspond to a group section then raise
 @code{E_inval}.
 
-@item get_group_signatures = string[]
+@item Elf64_File.get_group_signatures = string[]
+@itemx Elf32_File.get_group_signatures = string[]
 
 Return an array of strings with the signatures of the section groups
 present in this ELF file.
 
-@item get_section_group = (string @var{name}) Elf64_Shdr[]
-@itemx get_section_group = (string @var{name}) Elf32_Shdr[]
+@item Elf64_File.get_section_group = (string @var{name}) Elf64_Shdr[]
+@itemx Elf32_File.get_section_group = (string @var{name}) Elf32_Shdr[]
 
 Given the @var{name} of a section group, return an array with the
 section headers corresponding to all the sections in that group.  If
@@ -995,8 +924,87 @@ the given name doesn't identify a section group in the ELF 
file then
 return an empty array.
 @end table
 
+@section Usage
+
+@noindent
+Poking at an ELF file usually starts by opening some IO space and
+mapping a @code{Elf64_File} (or @code{Elf32_File}):
+
+@example
+(poke) .file /bin/ls
+(poke) var elf = Elf64_File @@ 0#B
+@end example
+
+@noindent
+Once mapped, we can access any of the above fields.  For example,
+let's see how many sections and segments this file has:
+
+@example
+(poke) elf.shdr'length
+30UL
+(poke) elf.phdr'length
+11UL
+@end example
+
+@noindent
+In case the file didn't have a program header table, which always
+happens with object files, we would have got an exception if we tried
+to access the absent field @code{phdr}:
+
+@example
+$ echo '' | gcc -c -xc -o foo.o -
+$ poke foo.o
+(poke) load elf
+(poke) (Elf64_File @@ 0#B).phdr
+unhandled invalid element exception
+@end example
+
+@subsection Working with sections
+
+Unlike in older object formats (like a.out for example) the sections
+present in ELF files are not fixed nor they have fixed pre-defined
+names: there can be any number of them (including none) and they can
+have any arbitrary name.  Also, more than one section in the file can
+have the same name.
+
+So when it comes to ELF files, the process to determine whether one or
+more section with a given name exists in the file is a bit laborious:
+one has to traverse the section header table, fetch the section names
+from whatever appropriate string table, etc.
+
+The following methods, that you can use in your own pickles, scripts,
+or at the prompt, are handy to look at particular sections in the
+file.
+
+@subsection Working with string tables
+
+The names of several entities in ELF files are stored in different
+string table, which are themselves stored in different sections.
+There are different rules establishing where exactly the name of
+certain entities (sections, symbols, ...) are to be found.
+
+These rules are not trivial and require traversing several data
+structures.  Therefore the @code{Elf64_File} (and @code{File32_File})
+type provides several methods in order to easily determine the name of
+these entities.
+
+@subsection Working with section groups
+
+ELF supports grouping several sections in a @dfn{section group}.  This
+is useful when several sections have to go together, because they rely
+on each other somehow.
+
+A section of type @code{SHT_GROUP} defines a section group.  Groups
+are univocally identified by a @dfn{group signature}, which is the
+name associated with a symbol that is stored in a particular symbol
+table, linked from the section header of the group defining section.
+
+Again, it is not exactly trivial to determine, for example, which of
+the sections in the ELF file pertain to which group.  Therefore the
+pickle provides the methods below:
+
 @node ELF Header
-@section ELF Header
+@chapter ELF Header
 @cindex @code{Elf32_Ehdr}
 @cindex @code{Elf64_Ehdr}
 
@@ -1009,6 +1017,8 @@ at headers directly is not that uncommon.
 The Poke types provided to denote ELF headers are @code{Elf64_Ehdr}
 and @code{Elf32_Ehdr}, for 64-bit and 32-bit ELF files respectively.
 
+@section Overview
+
 @example
 type Elf32_Ehdr =
   struct
@@ -1051,8 +1061,7 @@ type Elf64_Ehdr =
   @};
 @end example
 
-@noindent
-Where:
+@section Fields
 
 @table @code
 @item e_ident
@@ -1197,20 +1206,258 @@ uncommon but certainly possible) then this field 
contains
 @code{ELF_SHN_UNDEF}.
 @end table
 
-@node ELF Section Header Table
-@section ELF Section Header Table
+@section Usage
+
+XXX
+
+@node ELF Section Headers
+@chapter ELF Section Headers
 @cindex @code{Elf64_Shdr}
 @cindex @code{Elf32_Shdr}
 
-XXX
+Sections can be stored anywhere in an ELF file.  They can also be of
+any size, of any type, have any name (or no name) and their contents
+are free.  The ELF file therefore contains a table, called the
+@dfn{section header table}, whose entries describe each section.  This
+table is sized and linked from the ELF header via the @code{e_shoff}
+field.  As we have seen, the section header table is available in the
+@code{shdr} field of @code{Elf32_File} and @code{Elf64_File}.
+
+The Poke types denoting entries in the section header table are
+@code{Elf32_Shdr} and @code{Elf64_Shdr} for ELF32 and ELF64
+respectively.
+
+@section Overview
+
+@example
+type Elf32_Shdr =
+  struct
+  @{
+    offset<Elf_Word,B> sh_name;
+    Elf_Word sh_type;
+    Elf_Word sh_flags;
+    Elf32_Addr sh_addr;
+    Elf32_Off sh_offset;
+    offset<Elf_Word,B> sh_size;
+    Elf_Word sh_link;
+    Elf_Word sh_info;
+    Elf_Word sh_addralign;
+    offset<Elf_Word,B> sh_entsize;
+  @};
+@end example
+
+@example
+type Elf64_Shdr =
+  struct
+  @{
+    offset<Elf_Word,B> sh_name;
+    Elf_Word sh_type;
+    Elf64_Xword sh_flags;
+    Elf64_Addr sh_addr;
+    Elf64_Off sh_offset;
+    offset<Elf64_Xword,B> sh_size;
+    Elf_Word sh_link;
+    Elf_Word sh_info;
+    Elf64_Xword sh_addralign;
+    offset<Elf64_Xword,B> sh_entsize;
+  @};
+@end example
+
+@section Fields
+
+@table @code
+@item sh_name
+Is the offset to the name of this section in the file's section string
+table.  Two or more sections can share the same name.
+
+@item sh_type
+Is a code identifying the type of the section.  This is one of the
+@code{ELF_SHT_*} values.
+
+The type of a section determines what kind of contents (if any) a
+section has: relocations, a symbol table, a string table, executable
+compiled code, etc.  These are the types defined in the base spec:
+
+@table @code
+@item ELF_SHT_NULL
+This marks ``unused'' entry in the section header table.  The first
+entry in the table seems to always be an unused entry.  Unused entries
+have empty names.
+
+@item ELF_SHT_PROGBITS
+Section is what the spec calls ``program specific (private) data.''
+In practice, this basically means executable code.  The prototypical
+progbits section is @code{.text}.
+
+@item ELF_SHT_SYMTAB
+Section contains a symbol table.  Each symbol table is an array of
+@code{ELF64_Sym} (@code{Elf32_Sym} in ELF32) values spanning for
+@code{sh_size} bytes.  @xref{ELF Symbols}.
+
+@item ELF_SHT_STRTAB
+Section contains a string table.  Each string table is an array of
+NULL terminated strings spanning for @code{sh_size} bytes.
+
+@item ELF_SHT_RELA
+@itemx ELF_SHT_REL
+Section contains ELF relocations, with or without explicit addend.
+Each section contains an array of @code{Elf64_Rela} or
+@code{Elf64_Rel} (@code{Elf32_Rela} or @code{Elf32_Rel} in ELF32)
+values spanning for @code{sh_size} bytes.  @xref{ELF Relocations}.
+
+@item ELF_SHT_HASH
+Section contains a symbol hash table.
+
+@item ELF_SHT_DYNAMIC
+Section contains dynamic linking information in the form of a sequence
+of @dfn{dynamic tags}.  This is an array of @code{Elf64_Dyn}
+(@code{Elf32_Dyn} in ELF32) values spanning for @code{sh_size} bytes.
+@xref{ELF Dynamic Info}.
+
+@item ELF_SHT_NOTE
+Section contains @dfn{notes}.  These are flexible annotations that are
+usually used in order to reflect certain ``auxiliary'' attributes of
+the ELF file.  For example, the name and full version of the compiler
+that generated it.  The format in which the notes are encoded is well
+defined, and supported by the elf pickles.  @xref{ELF Notes}.
+
+@item ELF_SHT_SHLIB
+This value for @code{sh_type} is reserved by the ELF specification and
+has undefined semantics.
+
+@item ELF_SHT_DYNSYM
+
+@item ELF_SHT_NOBITS
+The section contents occupy no bits in the file.
+
+@item ELF_SHT_INIT_ARRAY
+@itemx ELF_SHT_FINI_ARRAY
+@itemx ELF_SHT_PREINIT_ARRAY
+Section contains an array of pointers to
+initialization/finalization/pre-initialization functions, which are
+parameter-less procedures that do not return any value.  This is an
+array of @code{offset<uint<64>,B>} (@code{offset<uint<32>,B>} in
+ELF32) values spanning for @code{sh_size} bytes.
+
+@item ELF_SHT_GROUP
+Section contains the definition of an ELF section group.  @xref{ELF
+File}.
+
+@item ELF_SHT_SYMTAB_SHNDX
+Section contains indices for @code{SHN_XINDEX} entries.
+@end table
 
-@node ELF Program Header Table
-@section ELF Program Header Table
+The ELF supplements for architectures/machines and operating systems
+introduce their own additional section types.  @xref{ELF Machines}.
+
+This field is checked against the @code{section-types} configuration
+parameter, and pretty-printed accordingly.
+
+@item sh_flags
+Is a bitmap where each enabled bit flags some particular property of
+the section.  This is one of the @code{ELF_SHF_*} values.  These are
+the flags defined in the base spec:
+
+@table @code
+@item ELF_SHF_WRITE
+The section contains data that should be writable during process
+execution.
+
+@item ELF_SHF_ALLOC
+The section contents are actually loaded into memory during process
+execution.
+
+@item ELF_SHF_EXECINSTR
+The section contains executable machine instructions.
+
+@item ELF_SHF_MERGE
+The section contents can be merged to eliminate duplication.  The ELF
+spec provides an algorithm (to be implemented by link editors) that
+explains how to merge sections flagged with this flag.  The algorithm
+covers two cases: merge-able sections containing elements of fixed
+size, and string tables.
+
+@item ELF_SHF_STRINGS
+The section contains a string table.
+
+@item ELF_SHF_INFO_LINK
+The @code{sh_info} field of this section header contains a section
+header table index.
+
+@item ELF_SHF_LINK_ORDER
+This section is to be ordered in a particular way by link editors.
+The order to use is specified by a link to other section header table
+via @code{sh_info}.  See the ELF spec for details.
+
+@item ELF_SHF_OS_NONCONFORMING
+This section requires special OS support to be linked.
+
+@item ELF_SHF_OS_TLS
+This section holds @dfn{thread-local storage}.
+
+@item ELF_SHF_COMPRESSED
+This section contents are compressed.  Sections flagged as compressed
+cannot have the flag @code{ELF_SHF_ALLOC} set.  Also, sections of type
+@code{ELF_SHT_NOBITS} cannot be compressed.
+@end table
+
+The ELF supplements for architectures/machines and operating systems
+introduce their own additional section types.  @xref{ELF Machines}.
+
+This field is checked against the @code{section-flags} configuration
+parameter, and pretty-printed accordingly.
+@end table
+
+@c @section Usage
+
+@c XXX accessing the section contents
+@c XXX listing section names
+@c XXX getting the section name
+@c XXX getting information about the section contents (elf_config.apropos)
+
+@node ELF Program Headers
+@chapter ELF Program Headers
 @cindex @code{Elf64_Phdr}
 @cindex @code{Elf32_Phdr}
 
 XXX
 
+@node ELF Symbols
+@chapter ELF Symbols
+@cindex @code{Elf64_Sym}
+@cindex @code{Elf32_Sym}
+
+XXX
+
+@node ELF Notes
+@chapter ELF Notes
+@cindex @code{Elf_Note}
+
+XXX
+
+@node ELF Relocations
+@chapter ELF Relocations
+@cindex @code{Elf64_RelInfo}
+@cindex @code{Elf64_Rel}
+@cindex @code{Elf64_Rela}
+@cindex @code{Elf32_RelInfo}
+@cindex @code{Elf32_Rel}
+@cindex @code{Elf32_Rela}
+
+XXX
+
+@node ELF Dynamic Info
+@chapter ELF Dynamic Info
+@cindex @code{Elf64_Dyn}
+@cindex @code{Elf32_Dyn}
+
+XXX
+
+@node ELF Machines
+@chapter ELF Machines
+
+XXX
+
 @node Indices
 @appendix Indices
 
-- 
2.30.2