Re: Erroneous YYBACKUP skeleton

[David Kastrup]

"Joel E. Denny" <address@hidden> writes:

> On Sun, 9 Oct 2011, David Kastrup wrote:
>
>> digging around on the web, I found an example of a failing YYBACKUP test
>> case in <URL:http://old.nabble.com/-PATCH--Update-TODO.-p20418035.html>
>> which has made it into the Bison TODO.  Further digging revealed
>> <URL:http://www.mail-archive.com/address@hidden/msg03249.html>.
>> 
>> Combining the information from both, the problem definitely appears to
>> be that YYBACKUP currently (as of bison 2.4.1, the version coming with
>> Ubuntu 11.10) does not correctly revert the state after popping the
>> stack.
>
> For the record, this test case has been failing since at least bison 
> 1.875, the oldest version I happen to have lying around.  1.875 was 
> released in 2003, so this is a very old bug for which I don't recall many 
> complaints.  It seems YYBACKUP isn't used much.

Likely.

> I'm a bit hesitant to extend the functionality of a feature that gets
> used so little.  Back when I was learning bison originally, I had read
> that YYBACKUP was a confusing feature that was best avoided.  I've
> never found motivation to use it.  Can you explain a bit why you want
> it?

Music functions in Lilypond can take a variety of typed arguments.  I
have recently implemented optional arguments.  The type of an argument
is specified by giving a Scheme predicate (evaluating to #f or #t).
Currently, a number of those predicates are hardwired into the parser,
so if you define your own predicates, they won't work all too well with
the parser.

Now for optional arguments, which are a new feature, one can place
tighter constraints on the syntax (like only accepting forms
recognizable without token lookahead).  Instead of having a large mess
where optional arguments are accepted (or not taken) by using operator
priorities in connection with hardwired syntax classes, an approach that
gets more and more useless the more syntax classes there are, I can
actually just call the Scheme predicate on the presumed argument, and if
it evaluates to false, I backup the presumed optional argument and enter
the rule for skipped optional arguments, so that the presumed optional
argument gets recognized as a later following mandatory argument.

Calling the Scheme predicate for deciding syntactic paths is a form of
semantical tie-in.  YYBACKUP appears to be a reasonably close fit.

> Well, if you're desperate, you could always define your own macro in
> your grammar file.

That is likely what I'll do in connection with autoconf checking for
both necessity as well as success of the patch.

>> I have not checked that the enhancement for reducing rules with more
>> than one token does actually work.  It seems like an obvious change,
>> but I don't really understand Bison enough to be sure.
>
> Does this mean you haven't found need for that enhancement then?

Up to now, I have been just doing basic tests and trying to figure out
why I could not get stuff to work at all.  It seems one should always be
able to program around the missing enhancement by creating one-element
rule aliases to all multi-element rules one wants to back up.  I see no
point in that encumbrance, though.  It is likely a relict from older
times (Yacc inheritage?)  times when YYPOPSTACK did not take an argument
and there were no nice ways to put it into a counted loop inside of a
macro.

> On Mon, 10 Oct 2011, David Kastrup wrote:
>
>> Adding yylen=0; after this line appears to be prudent as well, looking
>> at the rest of the code.
>
> At first glance, I don't see the need, but I guess it can't hurt.  Do you 
> have an example where it matters?

I've tried following the code paths somewhat, and for one thing other
code paths using YYPOPSTACK do take that measure.  For another, it would
appear that the error code paths are reachable after backup, and would
pop the stack again with the old recorded last rule depth that is no
longer connected to the reality of the parse stack.  This would likely
be a problem even for the yylen == 1 case.

-- 
David Kastrup