memoization and error messages

[Dirk Herrmann]

Hi folks,

since yesterday I have made some progress with the memoizer, I just feel
like showing you :-)))  I will show you some transcripts of how my
modified version of guile works, with some explanations.  The focus of
this mail is on error messages for syntax errors.  Part of this mail is
for showing you things that already work (and how they work), some parts
indicate open points.  In general, the handling of syntax errors is not
perfect yet, but its a start.


1) Syntactic keywords are no longer accepted as expressions.  Formerly,
guile would have accepted the following:
  guile> define
  #<macro 4020ada0>
But now, the correct position of syntactic keywords is checked by the
memoizer:
  guile> define
  ERROR: In procedure memoization:
  ERROR: Misplaced syntactic keyword define.
  ABORT: (syntax-error)

2) Error messages are, where possible, more explicit:
  guile> (if)
  ERROR: In procedure memoization:
  ERROR: In line 1: Missing or extra expression in (if).
  ABORT: (syntax-error)
  ***
  guile> (lambda (a a) #t)
  ERROR: In procedure memoization:
  ERROR: In line 2: Duplicate formal a in expression (lambda (a a) #t).
  ABORT: (syntax-error)
  ***
  guile> (lambda (1 a) #t)
  ERROR: In procedure memoization:
  ERROR: In line 3: Bad formal 1 in expression (lambda (1 a) #t).
  ABORT: (syntax-error)
  ***
  guile> () 
  ERROR: In procedure memoization:
  ERROR: Illegal empty combination ().
  ABORT: (syntax-error)
It should be noted, that checks for the empty combination () were formerly
done during evaluation.  Now, these checks are removed from the evaluator!
The bad thing with this example is, that information about line numbers
can't be issued for expressions that are no lists.  Since () is an
immediate value, there is no linenumber information available.  Currently,
it also doesn't work for the following example:
  guile> (foo () bar)
  ERROR: In procedure memoization:
  ERROR: Illegal empty combination ().
  ABORT: (syntax-error)
although here at least the line number of the outer expression could be
printed.  This, however, requires to carry a stack of outer expressions
around while doing memoization.  This is not implemented yet, but could
certainly be done.  I will add this feature eventually.
  ***
  guile> (foo . bar)
  ERROR: In procedure memoization:
  ERROR: In line 7: Bad expression (foo . bar).
  ABORT: (syntax-error)
Expressions are always checked to be proper lists during memoization.
Please note: The example above shows that if a linenumber is available it
is printed.

3) Bad expressions are detected before evaluating the code!
  guile> (define (foo) (foo . bar))
  ERROR: In procedure memoization:
  ERROR: In line 10: Bad expression (foo . bar).
  ABORT: (syntax-error)
Formerly, this error would have remained undetected until foo was actually
executed.

4) Backtraces are somewhat strange.  Compare the following backtraces:
  guile> (eval '(foo . bar) (interaction-environment))                
  Backtrace:
  In unknown file:
     ?: 0* [eval (foo . bar) #<directory (guile-user) 8051a40>]
  <unnamed port>: In procedure memoization in expression (eval (quote #) 
(interaction-environment)):
  <unnamed port>: In line 0: Bad expression (foo . bar).
  ABORT: (syntax-error)
*****
  guile> (eval '(if #t (foo . bar)) (interaction-environment))
  Backtrace:
  In unknown file:
     ?: 0* [eval (if #t (foo . bar)) #<directory (guile-user) 8051a40>]
     ?: 1* [if (if #t (foo . bar)) (#<eval-closure 40271520>)]
  <unnamed port>: In procedure memoization in expression (eval (quote #) 
(interaction-environment)):
  <unnamed port>: In line 1: Bad expression (foo . bar).
  ABORT: (syntax-error)
*****
  guile> (eval '(if #t (if #t (if #t (foo . bar)))) (interaction-environment))
  Backtrace:
  In unknown file:
     ?: 0* [eval (if #t (if #t (if #t #))) #<directory (guile-user) 8051a40>]
     ?: 1* [if (if #t (if #t (if #t #))) (#<eval-closure 40271520>)]
     ?: 2* [if (if #t (if #t (foo . bar))) (#<eval-closure 40271520>)]
     ?: 3* [if (if #t (foo . bar)) (#<eval-closure 40271520>)]
  ERROR: In procedure memoization:
  ERROR: In line 2: Bad expression (foo . bar).
  ABORT: (syntax-error)
What surprises me is that with every additional (if #t ...) I get another
backtrace frame displayed.  But, within the memoizer I am pretty sure that
I don't add backtrace information for the debugger.  If someone knows how
backtraces are computed, I would appreciate that information.


As you see, there are still some open points with respect to reporting
syntax errors.  But I also hope that you see that a separation of
memoization and execution also brings benefits with respect to early error
detection.  And, it should be obvious that the executor benefits from the
fact that all syntax checks are done in advance.

Please let me know if you can help in any of the open points above.  And,
certainly, if you have suggestions for improvement of the above style of
error reporting.

Best regards, 
Dirk Herrmann