Re: Make peg.el a built-in library?

[Eric Abrahamsen]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Basically a way of composing a parser out of smaller regexp-like
>   > expressions. They can be very useful in a wide variety of situations.
>
> It does sound useful.  Can you post a descripion of a specific simple
> example where this approach is advantageous?

I feel like I've ended up advocating for this thing when I know less
about it than anyone here, but...

My sense is that really powerful PEG systems are the sort of thing you
use to parse source code into ASTs, or do syntax highlighting, etc. We
don't need that, and the use-cases I have in mind, anyway, are simpler
situations where I want to parse a stream of
well-defined-but-still-pretty-complicated text. The sort of thing where
a regexp solution turns into a rat's nest very quickly.

One theoretical example is parsing IMAP server responses. The response
text is fully defined, but could vary enormously depending on the
capabilities of the server. Writing naive regexps is a headache.

Another non-theoretical example is the homemade token-parser in
lisp/gnus/gnus-search.el:390-680, which turns a string like:

from:bob (subject:lunch or subject:dinner)

into the sexp

((from . "bob") (or (subject . "lunch") (subject . "dinner"))

There are many, many libraries that need to do something similar. With
peg.el I can parse the above (including arbitrarily-nested
sub-expressions) with twenty lines of peg definition, which is
comprehensible to look at (once you've got the basics), easier to reason
about, and easier to modify. I guess it's sort of equivalent to a BNF.

PEGs and their implementation are the subject of academic research,
obviously, but for my modest uses, anyway, almost anything will do.