[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gawk-diffs] [SCM] gawk branch, master, updated. 472dc41713ee71609b2ddc8
|
From: |
Arnold Robbins |
|
Subject: |
[gawk-diffs] [SCM] gawk branch, master, updated. 472dc41713ee71609b2ddc8c77c00adb34354154 |
|
Date: |
Fri, 31 Aug 2012 11:52:33 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "gawk".
The branch, master has been updated
via 472dc41713ee71609b2ddc8c77c00adb34354154 (commit)
via 1dcb6bf5de886c0fdefa9e68557f348e3d5ec450 (commit)
from 9b8770d74f2e1cfd719fa0dbf21c676d1c64e8ea (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://git.sv.gnu.org/cgit/gawk.git/commit/?id=472dc41713ee71609b2ddc8c77c00adb34354154
commit 472dc41713ee71609b2ddc8c77c00adb34354154
Author: Arnold D. Robbins <address@hidden>
Date: Fri Aug 31 14:52:09 2012 +0300
Add api doc to repo during its development.
diff --git a/doc/api.texi b/doc/api.texi
new file mode 100644
index 0000000..77982ce
--- /dev/null
+++ b/doc/api.texi
@@ -0,0 +1,985 @@
+\input texinfo @c -*-texinfo-*-
address@hidden %**start of header (This is for running Texinfo on a region.)
address@hidden api.info
address@hidden Writing Extensions For Gawk
address@hidden %**end of header (This is for running Texinfo on a region.)
+
address@hidden Text creation and manipulation
address@hidden
+* Gawk: (gawk). A text scanning and processing language.
address@hidden direntry
address@hidden Individual utilities
address@hidden
+* awk: (gawk)Invoking gawk. Text scanning and processing.
address@hidden direntry
+
address@hidden xref-automatic-section-title
+
address@hidden The following information should be updated here only!
address@hidden This sets the edition of the document, the version of gawk it
address@hidden applies to and all the info about who's publishing this edition
+
address@hidden These apply across the board.
address@hidden UPDATE-MONTH August, 2012
address@hidden VERSION 4.1
address@hidden PATCHLEVEL 0
+
address@hidden FSF
+
address@hidden TITLE Writing Extensions for Gawk
address@hidden SUBTITLE A Temporary Manual
address@hidden EDITION 1
+
address@hidden
address@hidden DOCUMENT book
address@hidden CHAPTER chapter
address@hidden APPENDIX appendix
address@hidden SECTION section
address@hidden SUBSECTION subsection
address@hidden DARKCORNER @address@hidden,1cm}, @image{rflashlight,1cm}}
address@hidden COMMONEXT (c.e.)
address@hidden iftex
address@hidden
address@hidden DOCUMENT Info file
address@hidden CHAPTER major node
address@hidden APPENDIX major node
address@hidden SECTION minor node
address@hidden SUBSECTION node
address@hidden DARKCORNER (d.c.)
address@hidden COMMONEXT (c.e.)
address@hidden ifinfo
address@hidden
address@hidden DOCUMENT Web page
address@hidden CHAPTER chapter
address@hidden APPENDIX appendix
address@hidden SECTION section
address@hidden SUBSECTION subsection
address@hidden DARKCORNER (d.c.)
address@hidden COMMONEXT (c.e.)
address@hidden ifhtml
address@hidden
address@hidden DOCUMENT book
address@hidden CHAPTER chapter
address@hidden APPENDIX appendix
address@hidden SECTION section
address@hidden SUBSECTION subsection
address@hidden DARKCORNER (d.c.)
address@hidden COMMONEXT (c.e.)
address@hidden ifdocbook
address@hidden
address@hidden DOCUMENT book
address@hidden CHAPTER chapter
address@hidden APPENDIX appendix
address@hidden SECTION section
address@hidden SUBSECTION subsection
address@hidden DARKCORNER (d.c.)
address@hidden COMMONEXT (c.e.)
address@hidden ifplaintext
+
address@hidden some special symbols
address@hidden
address@hidden LEQ @address@hidden
address@hidden PI @address@hidden
address@hidden iftex
address@hidden
address@hidden LEQ <=
address@hidden PI @i{pi}
address@hidden ifnottex
+
address@hidden
address@hidden ii{text}
address@hidden
address@hidden macro
address@hidden ifnottex
+
address@hidden For HTML, spell out email addresses, to avoid problems with
address@hidden address harvesters for spammers.
address@hidden
address@hidden EMAIL{real,spelled}
+``\spelled\''
address@hidden macro
address@hidden ifhtml
address@hidden
address@hidden EMAIL{real,spelled}
address@hidden
address@hidden macro
address@hidden ifnothtml
+
address@hidden FN file name
address@hidden FFN File Name
address@hidden DF data file
address@hidden DDF Data File
address@hidden PVERSION version
address@hidden CTL Ctrl
+
address@hidden
+Some comments on the layout for TeX.
+1. Use at least texinfo.tex 2000-09-06.09
+2. I have done A LOT of work to make this look good. There are
address@hidden' commands
+ and use of address@hidden ... @end group' in a number of places. If you muck
+ with anything, it's your responsibility not to break the layout.
address@hidden ignore
+
address@hidden merge the function and variable indexes into the concept index
address@hidden
address@hidden fn cp
address@hidden vr cp
address@hidden ifinfo
address@hidden
address@hidden fn cp
address@hidden vr cp
address@hidden iftex
address@hidden
address@hidden fn cp
address@hidden vr cp
address@hidden ifxml
+
address@hidden If "finalout" is commented out, the printed output will show
address@hidden black boxes that mark lines that are too long. Thus, it is
address@hidden unwise to comment it out when running a master in case there are
address@hidden overfulls which are deemed okay.
+
address@hidden
address@hidden
address@hidden iftex
+
address@hidden
+Copyright @copyright{} 2012
+Free Software Foundation, Inc.
address@hidden 2
+
+This is Edition @value{EDITION} of @address@hidden: @value{SUBTITLE}},
+for the @address@hidden (or later) version of the GNU
+implementation of AWK.
+
+Permission is granted to copy, distribute and/or modify this document
+under the terms of the GNU Free Documentation License, Version 1.3 or
+any later version published by the Free Software Foundation; with the
+Invariant Sections being ``GNU General Public License'', the Front-Cover
+texts being (a) (see below), and with the Back-Cover Texts being (b)
+(see below). A copy of the license is included in the section entitled
+``GNU Free Documentation License''.
+
address@hidden a
address@hidden
+``A GNU Manual''
+
address@hidden
+``You have the freedom to
+copy and modify this GNU manual. Buying copies from the FSF
+supports it in developing GNU and promoting software freedom.''
address@hidden enumerate
address@hidden copying
+
address@hidden Comment out the "smallbook" for technical review. Saves
address@hidden considerable paper. Remember to turn it back on *before*
address@hidden starting the page-breaking work.
+
address@hidden 4/2002: Karl Berry recommends commenting out this and the
address@hidden address@hidden odd', and letting users use `texi2dvi -t'
address@hidden if they want to waste paper.
address@hidden @smallbook
+
+
address@hidden Uncomment this for the release. Leaving it off saves paper
address@hidden during editing and review.
address@hidden odd
+
address@hidden
address@hidden @value{TITLE}
address@hidden @value{SUBTITLE}
address@hidden Edition @value{EDITION}
address@hidden @value{UPDATE-MONTH}
address@hidden Arnold D. Robbins
address@hidden Efraim Yawitz
+
address@hidden Include the Distribution inside the titlepage environment so
address@hidden that headings are turned off. Headings on and off do not work.
+
address@hidden
address@hidden 0pt plus 1filll
+``To boldly go where no man has gone before'' is a
+Registered Trademark of Paramount Pictures Corporation. @*
address@hidden sorry, i couldn't resist
address@hidden 3
+Published by:
address@hidden 1
+
+Free Software Foundation @*
+51 Franklin Street, Fifth Floor @*
+Boston, MA 02110-1301 USA @*
+Phone: +1-617-542-5942 @*
+Fax: +1-617-542-2652 @*
+Email: @email{gnu@@gnu.org} @*
+URL: @uref{http://www.gnu.org/} @*
+
address@hidden This one is correct for gawk 3.1.0 from the FSF
+ISBN 1-882114-28-0 @*
address@hidden 2
address@hidden
address@hidden titlepage
+
address@hidden Extension API
address@hidden Writing Extensions for @command{gawk}
+
+This @value{CHAPTER} describes how to extend @command{gawk} using
+code written in C or C++. If you don't know anything about C
+programming, you can safely skip this @value{CHAPTER}, although you
+may wish to review the documentation on the extensions that come
+with @command{gawk} (@pxref{Extension Samples}).
+
address@hidden Extension Intro
address@hidden Introduction
+
+An @dfn{extension} (sometimes called a @dfn{plug-in}) is a piece of external
code
+that @command{gawk} can load at run-time to provide additional
+functionality, over and above the built-in capabilities described in
+the rest of this @value{DOCUMENT}.
+
+Extensions are useful because they allow you (of course) to extend
address@hidden's functionality. For example, they can provide access to
+system calls (such as @code{chdir()} to change directory) and to other
+C library routines that could be of use. As with most software,
+``the sky is the limit;'' if you can imagine something that you might
+want to do and can write in C or C++, you can write an extension to do it!
+
+Extensions are written in C or C++, using the @dfn{Application Programming
+Interface} (API) defined for this purpose by the @command{gawk}
+developers. The rest of this @value{CHAPTER} explains the design
+decisions behind the API, the facilities it provides and how to use
+them, and presents a small sample extension. In addition, it documents
+the sample extensions included in the @command{gawk} distribution.
+
address@hidden Extension Design
address@hidden Extension API Design
+
+The first version of extensions for @command{gawk}
+was developed in the mid-1990s and released with @command{gawk} 3.1 in
+the late 1990s. The basic mechanisms and design remained unchanged for
+close to 15 years, until 2012.
+
+The old extension mechanism used data types and functions from
address@hidden itself, with a ``clever hack'' to install extension
+functions.
+
address@hidden included some sample extensions, of which a few were
+really useful. However, it was clear from the outset that the extension
+mechanism was bolted onto the side and was not really thought out.
+
address@hidden Old Extension Problems
address@hidden Problems With The Old Mechanism
+
+The old extension mechanism had several problems:
+
address@hidden @bullet
address@hidden
+It depended heavily upon @command{gawk} internals.
+Any time the @code{NODE} structure changed,
+an extension would have to be recompiled. Furthermore, to really
+write extensions required understanding something about @command{gawk}'s
+internal functions. There was some documentation but it was quite minimal.
+
address@hidden
+Being able to call into @command{gawk} from an extension required linker
+facilities that are common on Unix-derived systems but that did
+not work on Windows systems; users wanting extensions on Windows
+had to statically link them into @command{gawk}, even though Windows supports
+dynamic loading of shared objects.
+
address@hidden
+The API would change occasionally as @command{gawk} changed; no compatibility
+between versions was ever offered or planned for.
address@hidden itemize
+
+Despite the drawbacks, the @command{xgawk} project developers forked
address@hidden and developed several significant extensions. They also
+enhanced @command{gawk}'s facilities relating to file inclusion and
+shared object access.
+
+A new API was desired for a long time, but only in 2012 did the
address@hidden maintainer and the @command{xgawk} developers finally
+start working on it together (FIXME: need more about @command{xgawk}).
+
address@hidden Extension New Mechansim Goals
address@hidden Goals For A New Mechansim
+
+Some goals for the new API were:
+
address@hidden @bullet
address@hidden
+The API should be independent of @command{gawk} internals. Changes in
address@hidden internals should not be visible to the writer of an
+extension function.
+
address@hidden
+The API should provide @emph{binary} compatibility across @command{gawk}
+releases as long as the API itself does not change.
+
address@hidden
+The API should enable extensions written in C to have roughly the
+same ``appearance'' as @command{awk} functions, meaning:
+
address@hidden @minus
address@hidden
+The ability to access function parameters.
+
address@hidden
+The ability to turn an undefined parameter into an array (call by reference).
+
address@hidden
+The ability to create, access and update global variables.
+
address@hidden
+It should provide
+easy access to all the elements of an array at once (``array flattening'')
+in order to loop over all the element in an easy fashion for C code.
address@hidden itemize
+
address@hidden
+The ability to create arrays (including @command{gawk}'s true
+multi-dimensional arrays).
+
address@hidden
+The API should use only features in ISO C 90, so that extensions
+can be written using the widest range of C and C++ compilers. The header
+should include the appropriate @samp{#ifdef __cplusplus} and @samp{extern "C"}
+magic so that a C++ compiler could be used. (If using the C++, the runtime
+system has to be smart enough to call any constructors and destructors,
+as @command{gawk} is a C program.)
+
address@hidden
+The API mechanism should not require access to @command{gawk}'s
address@hidden @dfn{symbols} are the variables and functions
+defined inside @command{gawk}. Access to these symbols by code
+external to @command{gawk} loaded dynamically at run-time, is
+problematic on Windows.} by the compile-time or dynamic linker,
+in order to enable creation of extensions that will also work on Windows.
address@hidden itemize
+
address@hidden Extension Other Design Decisions
address@hidden Other Design Decisions
+
+As an ``arbitrary'' design decision, extensions cannot access or change
+built-in variables and arrays (such as @code{ARGV}, @code{FS}), with
+the exception of @code{PROCINFO}. (Read-only access could in theory be
+allowed but wasn't.)
+
+The reason for this is to prevent an extension function from affecting
+the flow of an @command{awk} program outside its control. While a real
address@hidden function can do what it likes, that is at the discretion
+of the programmer. An extension function should provide a service or
+make a C API available for use within @command{awk}, and not mess with
address@hidden or @code{ARGC} and @code{ARGV}.
+
+In addition, it becomes easy to start down a slippery slope. How
+much access to @command{gawk} facilities do extensions need?
+Do they need @code{getline}? What about calling @code{gsub()} or
+compiling regular expressions? What about calling into @command{awk}
+functions? (@emph{That} would be messy.)
+
+In order to avoid these issues, the @command{gawk} developers chose
+to start with the simplest,
+most basic features that are still truly useful.
+
+Another decision is that
+although @command{gawk} provides nice things like MPFR, and arrays indexed
+internally by integers, we are not bringing these features
+out to the API in order to keep things simple and close to traditional
address@hidden semantics.
+
address@hidden Extension Mechanism Outline
address@hidden At A High Level How It Works
+
+The requirement to avoid access to @command{gawk}'s symbols is, at
+first glance, a difficult one to meet.
+
+One design, apparently used by Perl
+and Ruby and maybe others, would be to make the mainline @command{gawk} code
+into a library, with the @command{gawk} program a small @code{main()}
+linked against the library.
+
+This seemed like the tail wagging the dog, complicating build and
+installation and making a simple copy of the @command{gawk} executable
+from one system to another (or one place to another on the same
+system!) into a chancy operation.
+
+Pat Rankin suggested the solution that was adopted. Communication between
address@hidden and an extension is two way. First, when an extension
+is loaded, it is passed a pointer to a @code{struct} whose fields are
+function pointers.
+
+The extension can call functions inside @command{gawk} through these
+function pointers, at runtime, without needing (link time) access
+to @command{gawk}'s symbols. One of these function pointers is to a
+function for ``registering'' new built-in functions.
+
+In the other direction, the extension registers its new functions
+with @command{gawk} by passing function pointers to the functions that
+provide the new feature (@code{do_chdir()}, for example). @command{gawk}
+associates the function pointer with a name and can then call it, using
+a defined calling convention. The
address@hidden@var{xxx}()} function, in turn, then uses the function pointers in
+the API @code{struct} to do its work.
+
+Convenience macros in the @file{gawkapi.h} header file make calling through
+the function pointers look like regular function calls so that extension
+code is quite readable and understandable.
+
+Although all of this sounds medium complicated, the result is that
+extension code is quite clean and straightforward. This can be seen in
+the sample extensions @file{filefuncs.c} and also the @file{testext.c}
+code for testing the APIs.
+
+Some other bits and pieces:
+
address@hidden @bullet
address@hidden
+The API provides access to @command{gawk}'s @address@hidden values,
+reflecting command line options, like @code{do_lint}, @code{do_profiling}
+and so on (@pxref{Extension API Variables}).
+These are informational: an extension cannot affect these
+inside @command{gawk}. In addtion, attempting to assign to them
+produces a compile-time error.
+
address@hidden
+The API also provides major and minor version numbers, so that an
+extension can check if the @command{gawk} it is loaded with supports the
+facilties it was compiled with. (Version mismatches ``shouldn't''
+happen, but we all know how @emph{that} goes.)
+
address@hidden
+An extension may register a version string with @command{gawk}; this
+allows @command{gawk} to dump extension version information when
+invoked with the @option{--version} option.
address@hidden itemize
+
address@hidden Extension Future Grouth
address@hidden Room For Future Growth
+
+The API also provides room for future growth, in two ways.
+
+An ``extension id'' is passed into the extension when its loaded. This
+extension id is then passed back to @command{gawk} with each function
+call. This allows @command{gawk} to identify the extension calling it,
+should it need to know.
+
+A ``name space'' is passed into @command{gawk} when an extension
+is registered. This allows for some future mechanism for grouping
+extension functions and possibly avoiding name conflicts.
+
address@hidden Extension Versioning
address@hidden API Versioning
+
+The API provides both a ``major'' and a ``minor'' version number.
+The API versions are available at compile time as constants:
+
address@hidden @code
address@hidden GAWK_API_MAJOR_VERSION
+The major version of the API.
+
address@hidden GAWK_API_MINOR_VERSION
+The minor version of the API.
address@hidden table
+
+The minor version increases when new functions are added to the API. Such
+new functions are always added to the end of the API @code{struct}.
+
+The major version increases (and minor version is reset to zero) if any
+of the data types change size or member order, or if any of the existing
+functions change signature.
+
+It could happen that
+an extension may be compiled against one version of the API but loaded
+by a version of @command{gawk} using a different version. For this
+reason, the major and minor API versions of the running @command{gawk}
+are included in the API @code{struct} as read-only constant integers:
+
address@hidden @code
address@hidden api->major_version
+The major version of the running @command{gawk}.
+
address@hidden api->minor_version
+The minor version of the running @command{gawk}.
address@hidden table
+
+It is up to the extension to decide if there are API incompatibilities.
+Typically a check like this is enough:
+
address@hidden
+if (api->major_version != GAWK_API_MAJOR_VERSION
+ || api->minor_version < GAWK_API_MINOR_VERSION) @{
+ fprintf(stderr, "foo_extension: version mismatch with gawk!\n");
+ fprintf(stderr, "\tmy version (%d, %d), gawk version (%d, %d)\n",
+ GAWK_API_MAJOR_VERSION, GAWK_API_MINOR_VERSION,
+ api->major_version, api->minor_version);
+ exit(1);
address@hidden
address@hidden example
+
+Such code is included in the boilerplate @code{dl_load_func} macro
+provided in @file{gawkapi.h}.
+
address@hidden Extension API Description
address@hidden API Description
+
address@hidden Efraim: Here is where you get to start working! :-)
+
address@hidden this is just a point that should be included in the discussion
+As the API has evolved, it has settled into a pattern where
+query routines return an @code{awk_bool_t}, with ``true'' meaning success and
+``false'' not, but a false return still fills in the actual type.
+
address@hidden Extension API Data Types
address@hidden Data Types
+
address@hidden Extension API Functions
address@hidden Functions
+
+Access to facilities within @command{gawk} are made available
+by calling through function pointers passed into your extension.
+
+While you may call through these function pointers directly,
+the interface is not so pretty. To make extension code look
+more like regular code, the @file{gawkapi.h} header
+file defines a number of macros which you should use in your code.
+This section presents the macros as if they were functions.
+
+API function pointers are provided for the following kinds of operations:
+
address@hidden @bullet
address@hidden
+Accessing parameters, including converting an undefined paramater into
+array
+
address@hidden
+Printing fatal, warning, and lint warning messages
+
address@hidden
+Registering input parsers, output wrappers, and two-way processors
+
address@hidden
+Updating @code{ERRNO}, or unsetting it
+
address@hidden
+Registering an extension function
+
address@hidden
+Registering exit handler functions to be called with @command{gawk} exits
+
address@hidden
+Accessing and creating global variables
+
address@hidden
+Symbol table access: retreiving a global variable, creating one, or changing
one.
+This also includes the ability to create a scalar variable that will be
@emph{constant}
+within @command{awk} code.
+
address@hidden
+Manipulating arrays
address@hidden @minus
address@hidden
+Retrieving, adding, deleting, and modifying elements
address@hidden
+Getting the count of elements in an array
address@hidden
+Creating a new array
address@hidden
+Clearing an array
address@hidden
+Flattening an array for easy C style looping over an array
address@hidden itemize
+
address@hidden
+Creating and releasing cached values; this provides an
+efficient way to use values for multiple variables and
+can be a big performance win.
+
address@hidden
+Registering an informational version string.
address@hidden itemize
+
+Points about using the API:
+
address@hidden @item
+
+In general, all pointers filled in by @command{gawk} are to memory
+managed by @command{gawk} and should be treated by the extension as
+read-only. Memory for @emph{all} strings passed into @command{gawk}
+from the extension @emph{must} come from @code{malloc()} and is managed
+by @command{gawk} from then on.
+
address@hidden @item
+
+The API defines several simple structs that map values as seen
+from @command{awk}. A value can be a @code{double}, a string, or an
+array (as in multidimensional arrays, or when creating a new array).
+Strings maintain both pointer and length since embedded @code{NUL}
+characters are allowed.
+
+By intent, strings are maintained using the current multibyte encoding (as
+defined by @address@hidden environment variables) and not using wide
+characters. This matches how @command{gawk} stores strings internally
+and also how characters are likely to be input and output from files.
+
+
address@hidden @item
+
+When retrieving a value (such as a parameter or that of a global variable
+or array element), the extension requests a specific type (number, string,
address@hidden FIXME: expand to include scalars, value cookies
+array, or ``undefined''). When the request is undefined, the returned value
+will have the real underlying type.
+
+However, if the request and actual type don't match, the access function
+returns ``false'' and fills in the type of the actual value that is there,
+so that the extension can, e.g., print an error message
+(``scalar passed where array expected'').
+
address@hidden This is documented in the header file and needs some expanding
upon.
address@hidden The table there should be presented here
+
address@hidden Input Parsers
address@hidden Customized Input Parsers
+
+By default, @command{gawk} reads text files as its input. It uses the value
+of @code{RS} to find the end of the record, and then uses @code{FS}
+(or @code{FIELDWIDTHS}) to split it into fields. Additionally, it sets
+the value of @code{RT}. (FIXME: pxrefs as needed.)
+
+If you want, you can provide your own, custom, input parser. An input
+parser's job is to return a record to the @command{gawk} record processing
+code, along with indicators for the value and length of the data to be
+used for @code{RT}, if any.
+
+To provide an input parser, you must provide two functions
+(where @var{XXX} is a prefix name for your extension):
+
address@hidden @code
address@hidden int @var{XXX}_can_take_file(const IOBUF_PUBLIC *iobuf)
+This function examines the information available in @code{iobuf}
+(which we discuss shortly). Based on the information there, it
+decides if the input parser should be used for this file.
+If so, it should return true (non-zero). Otherwise, it should
+return false (zero).
+
address@hidden int @var{XXX}_take_control_of(IOBUF_PUBLIC *iobuf)
+When @command{gawk} decides to hand control of the file over to the
+input parser, it calls this function. This function in turn must fill
+in certain fields in the @code{IOBUF_PUBLIC} structure, and ensure
+that certain conditions are true. It should then return true. If an
+error of some kind occurs, it should not fill in any fields, and should
+return false; then @command{gawk} will not use the input parser.
+The details are presented shortly.
address@hidden table
+
+Your extension should package these functions inside an
address@hidden, which looks like this (from @file{gawkapi.h}):
+
address@hidden
+typedef struct input_parser @{
+ const char *name; /* name of parser */
+ int (*can_take_file)(const IOBUF_PUBLIC *iobuf);
+ int (*take_control_of)(IOBUF_PUBLIC *iobuf);
+ struct input_parser *awk_const next; /* for use by gawk */
address@hidden awk_input_parser_t;
address@hidden example
+
+The steps are as follows:
+
address@hidden
address@hidden
+Create a @code{static awk_input_parser_t} variable and initialize it
+appropriately.
+
address@hidden
+When your extension is loaded, register your input parser with
address@hidden using the @code{register_input_parser()} API.
address@hidden enumerate
+
+An @code{IOBUF_PUBLIC} looks like this:
+
address@hidden
+typedef struct iobuf_public @{
+ const char *name; /* filename */
+ int fd; /* file descriptor */
+#define INVALID_HANDLE (-1)
+ void *opaque; /* private data for input parsers */
+ /*
+ * The get_record function is called to read the next record of data.
+ * It should return the length of the input record (or EOF), and
+ * it should set *out to point to the contents of $0. Note that
+ * gawk will make a copy of the record in *out, so the parser is
+ * responsible for managing its own memory buffer. If an error
+ * occurs, the function should return EOF and set *errcode
+ * to a non-zero value. In that case, if *errcode does not equal
+ * -1, gawk will automatically update the ERRNO variable based on
+ * the value of *errcode (e.g. setting *errcode = errno should do
+ * the right thing). It is guaranteed that errcode is a valid
+ * pointer, so there is no need to test for a NULL value. The
+ * caller sets *errcode to 0, so there is no need to set it unless
+ * an error occurs. The rt_start and rt_len arguments should be
+ * used to return RT to gawk. Gawk will make its own copy of RT,
+ * so the parser is responsible for managing this memory. If EOF is
+ * not returned, the parser must set *rt_len (and *rt_start if *rt_len
+ * is non-zero).
+ */
+ int (*get_record)(char **out, struct iobuf_public *, int *errcode,
+ char **rt_start, size_t *rt_len);
+ /*
+ * The close_func is called to allow the parser to free private data.
+ * Gawk itself will close the fd unless close_func sets it to -1.
+ */
+ void (*close_func)(struct iobuf_public *);
+
+ /* put last, for alignment. bleah */
+ struct stat sbuf; /* stat buf */
address@hidden IOBUF_PUBLIC;
address@hidden example
+
+The fields can be divided into two categories: those for use (initially,
+at least) by @address@hidden()}, and those for use by
address@hidden@var{XXX}_take_control_of()}. The first group of fields and
their uses
+are as follows:
+
address@hidden @code
address@hidden const char *name;
+The name of the file.
+
address@hidden int fd;
+A file descriptor for the file. If @command{gawk} was able to
+open the file, then it will @emph{not} be equal to
address@hidden Otherwise, it will.
+
address@hidden struct stat sbuf;
+If file descriptor is valid, then @command{gawk} will have filled
+in this structure with a call to the @code{fstat()} system call.
address@hidden table
+
+The @address@hidden()} function should examine these
+fields and decide if the input parser should be used for the file.
+The decision can be made based upon @command{gawk} state (the value
+of a variable defined previously by the extension and set by
address@hidden code), the name of the
+file, whether or not the file descriptor is valid, the information
+in the @code{struct stat}, or any combination of the above.
+
+Once @address@hidden()} has returned true, and
address@hidden has decided to use your input parser, it will call
address@hidden@var{XXX}_take_control_of()}. That function then fills in at
+least the @code{get_record} field of the @code{IOBUF_PUBLIC}. It must
+also ensure that @code{fd} is not set to @code{INVALID_HANDLE}. All of
+the fields that may be filled by @address@hidden()}
+are as follows:
+
address@hidden @code
address@hidden void *opaque;
+This is used to hold any state information needed by the input parser
+for this file. It is ``opaque'' to @command{gawk}. The input parser
+is not required to use this pointer.
+
address@hidden int (*get_record)(char **out, struct iobuf_public *, int
*errcode,
address@hidden char **rt_start, size_t *rt_len);
+This is a function pointer that should be set to point to the
+function that creates the input records.
+Said function is the core of the input parser. Its behavior is
+described below.
+
address@hidden void (*close_func)(struct iobuf_public *);
+This is a function pointer that should be set to point to the
+function that does the ``tear down.'' It should release any resources
+allocated by @address@hidden()}. It may also close
+the file. If it does so, it shold set the @code{fd} field to
address@hidden
+
+Having a ``tear down'' function is optional. If your input parser does
+not need it, do not set this field.
address@hidden table
+
+The @address@hidden()} function does the work of creating
+input records. The parameters are as follows:
+
address@hidden @code
address@hidden char **out
+This is a pointer to a @code{char *} variable which is set to point
+to the record. @command{gawk} will make its own copy of the data, so
+it is the responsibility of the extension to manage this storage.
+
address@hidden struct iobuf_public *iobuf
+This is the @code{IOBUF_PUBLIC} for the file. The fields should be
+used for reading data (@code{fd}) and for managing private state
+(@code{opaque}), if any.
+
address@hidden int *errcode
+If an error occurs, @code{*errcode} should be set to an appropriate
+code from @code{<errno.h>}.
+
address@hidden char **rt_start
address@hidden size_t *rt_len
+If the concept of a ``record terminator'' makes sense, then
address@hidden should be set to point to the data to be used for
address@hidden, and @code{*rt_len} should be set to the length of the
+data. Otherwise, @code{*rt_len} should be set to zero.
address@hidden table
+
+The return value is the length of the buffer pointed to by
address@hidden, or @code{EOF} if end-of-file was reached or an
+error occurred.
+
address@hidden ships with a sample extension (@pxref{Extension Sample
+Readdir}) that reads directories, returning records for each entry in
+the directory. You may wish to use that code as a guide for writing
+your own input parser.
+
+When writing an input parser, you should think about (and document)
+how it is expected to interact with @command{awk} code. You may want
+it to always be called, and take effect as appropriate (as the
address@hidden extension does). Or you may want it to take effect
+based upon the value of an @code{awk} variable, as the XML extension
+from the @code{gawkextlib} project does (@pxref{gawkextlib}).
+In the latter case, code in a @code{BEGINFILE} section (FIXME: pxref)
+can look at @code{FILENAME} and @code{ERRNO} to decide whether or
+not to activate an input parser.
+
address@hidden Output Wrappers
address@hidden Customized Output Wrappers
+
address@hidden Two-way processors
address@hidden Customized Two-way Processors
+
address@hidden Extension API Variables
address@hidden External Variables
+
+The API provides access to several variables that describe
+whether the corresponding command-line options were enabled when
address@hidden was invoked. The variables are:
+
address@hidden @code
address@hidden do_lint
+This variable will be true if the @option{--lint} option was passed
+(FIXME: pxref).
+
address@hidden do_traditional
+This variable will be true if the @option{--traditional} option was passed.
+
address@hidden do_profile
+This variable will be true if the @option{--profile} option was passed.
+
address@hidden do_sandbox
+This variable will be true if the @option{--sandbox} option was passed.
+
address@hidden do_debug
+This variable will be true if the @option{--debug} option was passed.
+
address@hidden do_mpfr
+This variable will be true if the @option{--bignum} option was passed.
address@hidden table
+
+The value of @code{do_lint} can change if @command{awk} code
+modifies the @code{LINT} built-in variable (FIXME: pxref).
+The others should not change during execution.
+
address@hidden Extension API Boilerplate
address@hidden Boilerplate Code
+
address@hidden Extension Example
address@hidden Example: Some File Functions
+
address@hidden It's enough to show chdir and stat, no need for fts
+
address@hidden Extension Samples
address@hidden Sample Extensions
+
address@hidden Extension Sample File Functions
address@hidden File Related Functions
+
address@hidden can pull doc from man pages in extension directory
+
address@hidden Extension Sample Fnmatch
address@hidden Interface To @code{fnmatch()}
+
address@hidden Extension Sample Fork
address@hidden Interface to @code{fork()}, @code{wait()} and @code{waitpid()}
+
address@hidden Extension Sample Ord
address@hidden Character and Numeric values: @code{ord()} and @code{chr()}
+
address@hidden Extension Sample Readdir
address@hidden Reading Directories
+
address@hidden Extension Sample Readfile
address@hidden Reading An Entire File
+
address@hidden Extension Sample Read write array
address@hidden Dumping and Restoring An Array
+
address@hidden Extension Sample API Tests
address@hidden API Tests
+
address@hidden Extension Sample Time
address@hidden Time Functions
+
address@hidden time
address@hidden sleep
+
+These functions can be used by either invoking @command{gawk}
+with a command-line argument of @option{-l time} or by
+inserting @code{@@load "time"} in your script.
+
address@hidden @code
+
address@hidden @code{gettimeofday} time extension function
address@hidden gettimeofday()
+This function returns the time that has elapsed since 1970-01-01 UTC
+as a floating point value. It should have sub-second precision, but
+the actual precision will vary based on the platform. If the time
+is unavailable on this platform, it returns @minus{}1 and sets @code{ERRNO}.
+If the standard C @code{gettimeofday()} system call is available on this
platform,
+then it simply returns the value. Otherwise, if on Windows,
+it tries to use @code{GetSystemTimeAsFileTime()}.
+
address@hidden @code{sleep} time extension function
address@hidden sleep(@var{seconds})
+This function attempts to sleep for @var{seconds} seconds.
+Note that @var{seconds} may be a floating-point (non-integral) value.
+If @var{seconds} is negative, or the attempt to sleep fails,
+then it returns @minus{}1 and sets @code{ERRNO}.
+Otherwise, the function should return 0 after sleeping
+for the indicated amount of time. Implementation details: depending
+on platform availability, it tries to use @code{nanosleep()} or @code{select()}
+to implement the delay.
+
address@hidden table
+
address@hidden gawkextlib
address@hidden The @code{gawkextlib} Project
+
+The @uref{http://sourceforge.net/projects/gawkextlib/, @code{gawkextlib}}
project
+provides a number of @command{gawk} extensions, including one for
+processing XML files. This is the evolution of the original @command{xgawk}
+(XML @command{gawk}) project.
+
+As of this writing, there are four extensions:
+
address@hidden @bullet
address@hidden
+XML parser extension, using the @uref{http://expat.sourceforge.net, Expat}
+XML parsing library
+
address@hidden
+Postgres SQL extension
+
address@hidden
+GD graphics library extension
+
address@hidden
+MPFR library extension.
+This provides access to a number of MPFR functions which @command{gawk}'s
+native MPFR support does not.
address@hidden itemize
+
+The @code{time} extension described earlier
+(@pxref{Extension Sample Time})
+was originally from this project but has been moved in to the
+main @command{gawk} distribution.
+
address@hidden
http://git.sv.gnu.org/cgit/gawk.git/commit/?id=1dcb6bf5de886c0fdefa9e68557f348e3d5ec450
commit 1dcb6bf5de886c0fdefa9e68557f348e3d5ec450
Author: Arnold D. Robbins <address@hidden>
Date: Fri Aug 31 14:51:53 2012 +0300
Improve readdir extension.
diff --git a/extension/ChangeLog b/extension/ChangeLog
index 899f72d..9b5affc 100644
--- a/extension/ChangeLog
+++ b/extension/ChangeLog
@@ -1,3 +1,9 @@
+2012-08-28 Andrew J. Schorr <address@hidden>
+
+ * readdir.c: Have three states, 0, 1, 2 for never, fallback, and
+ always.
+ * readdir.3am: Adjust appropriately.
+
2012-08-29 Arnold D. Robbins <address@hidden>
Make fts work everywhere by using our own source.
diff --git a/extension/readdir.3am b/extension/readdir.3am
index 9cb4862..e7c6ee4 100644
--- a/extension/readdir.3am
+++ b/extension/readdir.3am
@@ -5,7 +5,7 @@ readdir \- directory input parser for gawk
.ft CW
@load "readdir"
.sp
-readdir_do_ftype(1) # or 0
+readdir_do_ftype(2) # or 0 or 1
.ft R
.SH DESCRIPTION
The
@@ -43,12 +43,16 @@ for a socket, and
(unknown) for anything else.
.PP
On systems without the file type information, calling
-.B readdir_do_ftype(1)
+.B readdir_do_ftype(2)
causes the extension to use
.IR stat (2)
to retrieve the appropriate information. This is not the default, since
.IR stat (2)
-is a potentially expensive operation.
+is a potentially expensive operation. By calling
+.B readdir_do_ftype(0)
+one can ensure that the file type
+information is never displayed, even when readily available in the
+directory entry.
.SH NOTES
On GNU/Linux systems, there are filesystems that don't support the
.B d_type
@@ -57,7 +61,7 @@ entry (see
and so the file type is always
.BR u .
Therefore, using
-.B readdir_do_ftype(1)
+.B readdir_do_ftype(2)
is advisable even on GNU/Linux systems. In this case, the
.I readdir
extension will fall back to using
diff --git a/extension/readdir.c b/extension/readdir.c
index 220d172..14039c8 100644
--- a/extension/readdir.c
+++ b/extension/readdir.c
@@ -62,7 +62,18 @@ static awk_bool_t (*init_func)(void) = init_readdir;
int plugin_is_GPL_compatible;
-static int do_ftype;
+/*
+ * ftype <= 0: never return file type info
+ * ftype == 1: return file type info only if it is available in dirent
+ * ftype >= 2: always return file type info, calling fstat if necessary
+ */
+static int do_ftype =
+#ifdef DT_BLK
+ 1
+#else
+ 0
+#endif
+ ;
/* ftype --- return type of file as a single character string */
@@ -91,7 +102,15 @@ ftype(struct dirent *entry)
}
#endif
- if (! do_ftype || lstat(entry->d_name, & sbuf) < 0)
+ if (do_ftype < 2)
+ /*
+ * Avoid "/u" since user did not insist on file type info,
+ * and it does not seem to be supported by dirent on this
+ * filesystem.
+ */
+ return NULL;
+
+ if (lstat(entry->d_name, & sbuf) < 0)
return "u";
switch (sbuf.st_mode & S_IFMT) {
@@ -157,8 +176,11 @@ dir_get_record(char **out, struct iobuf_public *iobuf, int
*errcode,
len = sprintf(the_dir->buf, "%llu/%s",
(unsigned long long) dirent->d_ino,
dirent->d_name);
- if (do_ftype)
- len += sprintf(the_dir->buf + len, "/%s", ftype(dirent));
+ if (do_ftype > 0) {
+ const char *ftstr = ftype(dirent);
+ if (ftstr)
+ len += sprintf(the_dir->buf + len, "/%s", ftstr);
+ }
*out = the_dir->buf;
@@ -286,7 +308,7 @@ do_readdir_do_ftype(int nargs, awk_value_t *result)
goto out;
}
- do_ftype = (flag.num_value != 0.0);
+ do_ftype = flag.num_value;
out:
return result;
diff --git a/test/readdir.awk b/test/readdir.awk
index 4d35be6..3ec664d 100644
--- a/test/readdir.awk
+++ b/test/readdir.awk
@@ -1,7 +1,7 @@
@load "readdir"
BEGIN {
- readdir_do_ftype(1)
+ readdir_do_ftype(2)
}
{ print }
-----------------------------------------------------------------------
Summary of changes:
doc/api.texi | 985 +++++++++++++++++++++++++++++++++++++++++++++++++
extension/ChangeLog | 6 +
extension/readdir.3am | 12 +-
extension/readdir.c | 32 ++-
test/readdir.awk | 2 +-
5 files changed, 1027 insertions(+), 10 deletions(-)
create mode 100644 doc/api.texi
hooks/post-receive
--
gawk
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [gawk-diffs] [SCM] gawk branch, master, updated. 472dc41713ee71609b2ddc8c77c00adb34354154,
Arnold Robbins <=