>> > Once the VM supports proper tail recursion, it's straightforward to
>> > generate automata that never perform a function call, at least not as part
>> > of the recognizer.
>>
>> It was straightforward beforehand as well (using a `while` loop instead
>> of recursion). And if you do use recursion, then it's not very much
>> simpler with `lexical-binding` than without because you still have to
>> take into account the possibility that the function gets redefined
>> during your recursion :-(
>>
>
> I think you're mistaking self-tail recursion for tail recursion.
No, I was simply restricting the discussion to the case you mention of
"generat[ing an] automata", in which case you usually have enough
control over the generated code to use a `while` loop if desired.
It's true you can avoid funcall dispatch overhead that way, but unless I'm missing something you are stuck with the overhead of the while loop plus whatever conditional branching mechanism you would use for dispatching to a state label, as opposed to simply jumping to the contents of a register. I'm not 100% on whether there's an ELisp construct that the byte compiler can turn into byte code that uses a table of labels for condirional dispatch the way a switch statement may be implemented in C.
Lynn