Re: Make peg.el a built-in library?

[Eric Abrahamsen]

On 08/26/21 08:34 AM, Eric Abrahamsen wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>>> Date: Wed, 25 Aug 2021 11:52:00 -0700
>>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>
>>> 
>>> In my on-again-off-again quest to not have to write text parsers myself,
>>> I was pointed towards the PEG library (in ELPA), which does pretty much
>>> exactly what I want (Parsing Expression Grammars).
>>> 
>>> Would the maintainers consider moving this into Emacs proper? I ask
>>> mostly because this would be very useful to have in Gnus, both to
>>> replace the home-made parser in gnus-search.el, and I would hope to
>>> parse eg IMAP server responses more fully and reliably.
>>
>> Fine with me, but please update the (outdated) Wiki page to say where
>> the latest peg.el is, when it is imported.
>
> Will do. Stefan also asked me to make sure the library actually does
> what I expect it to do, before making this move, so I'll write the code
> first.

Okay, I wrote some code: the "use-peg-in-gnus-search.diff" attachment is
the result of that. It works really well! A net removal of ~100 LOC
(obviously we're still in deficit with the addition of peg.el), it
already fixes some wrong behavior of the old parser, and it's much
easier to reason about and add new behavior to. It's the shiny
declarative future I was looking forward to.

Whether or not PEG gets added to core I'd like to propose some patches.
The "peg-doc-patches.diff" attachment adds some documentation to the
Commentary section, including an example grammar based on a
much-simplified version of what gnus-search does.

The peg-allow-symbols patch is more tentative. The issue is that _all_
of the entry-points to peg code are macros, meaning you can't build your
grammar up in a variable, and then pass that variable to any of
`peg-run', `peg-parse', `with-peg-rules', etc. Nobody will evaluate the
variable; you have to literally write the rules inside the
`with-peg-rules' form. It seems like a fairly plausible use-case to
store the rules in a variable or an option, even if you're not doing
run-time manipulation of them. The only solution, as Adam found with
org-ql, is to `eval' one of the macros.

This doesn't seem necessary! The patch has `with-peg-rules' check if the
rules are a symbol, and take the `symbol-value' if so. But I wonder if
it wouldn't be nicer to break some of the code out: `peg-normalize'
seems to be the entry-point for "compile this grammar", and that could
be modified to work the way that some languages provide for pre-compiled
regexps: a way to let the developer build and compile the grammar at
load-time or launch-time, then feed the stored compiled version to
parsing routines.

`peg-parse' could be a function, or maybe it also could also just check
if its argument is a symbol.

I hope someone will have some thoughts on this!

Eric

use-peg-in-gnus-search.diff
Description: Text Data

peg-doc-patch.diff
Description: Text Data

peg-allow-symbols.diff
Description: Text Data