Making TRs with 'guile-number and 'whitespace

[Phil]

Hi,

I'm hoping for a bit of help answering the 2 questions I have below on
guile-reader usage.

I'm trying out a solution to a puzzle, which takes as an input a series of
function calls with arguments, which are executed in turn.

The function definitions are in Guile and are trivial/irrelevant to the
issue below.

The tricky thing is that the input file is not in scheme format and so cannot be
trivially loaded - function calls look like this

function arg

Rather than:

(function arg)

Given the lack of s-exps in the input, it seemed I was going to get nowhere 
solving
this with a macro, but writing a new reader seemed like a possibility,
and guile-reader seemed like a sensible way of doing this.  (Obviously I
could solve the problem using read-line and a case statement, or just
the sed the original input file, but I'm interested in learning more
about DSLs, the reader, and macros).

See the code at the bottom of this e-mail which is doing more-or-less
what I want, but I've hit 2 ugly snags I'd like to understand better.

The code works by defining a new reader to load the input file, which
consists of 2 token readers - one matches any line starting with a
lowercase character (assumed within this restricted example to always be
a function call), and another which matches and ignores any newlines, etc.

In the lowercase-matching token reader I can't seem to use the Token Reader
Library
(https://www.nongnu.org/guile-reader/doc/guile-reader.html#Token-Reader-Library)
for reading whitespace or numerical values:

1. There should be a "whitespace" Token Reader - however in my function
ws-read, if I replace 'character with 'whitespace, ws-read is set to #f,
suggesting no such reader procedure exists?

2. In my function num-read if I replace 'character with 'guile-number
the returned symbols contain the numerical argument, however it is
prefixed with the first character of the function name, for example:

forward 9

Correctly returns from fn-read:

forward

But for num-read I get an extra "f" returned:

f9

(ws-read returns the space in-between as expected).

Hence to make num-read parse correctly, I've used a character code
instead of an integer parse, which is a bit of a hack, and only works for
single digit numbers.


Can anyone explain what is wrong with my use of 'guile-number and
'whitespace token library readers?


For point 1 I've had a look at the token-readers.c and it may be that there
is no whitepace TR?

For point 2 my guess is that it has something to do with the
make-token-reader function absorbing the first char, but I don't
understand why that's then appearing as part of the 3 parse in my-token-reader?

Also if my approach is daft, please feel free to suggest how else to
load the input.txt directly - although I am keen to understand what I'm
doing wrong in the code below and why, even if the approach isn't
brilliant the point of the exercise was to get familiar with guile-reader!


Thanks,
Phil.


I'm running on Guile 3.0.7 / guile-reader 0.6.3.

Full Puzzle Details (not really necessary to understand the problem, but
for reference) - https://adventofcode.com/2021/day/2


Example input.txt:
------------------

Full input is here:
https://github.com/falloutphil/aoc_2021/blob/main/02/input.txt

Short Example:
forward 9
down 8
down 2
down 4
up 8
forward 8
down 7
forward 3
down 8
down 9



Example code:
-------------

Code is here:
https://github.com/falloutphil/aoc_2021/blob/main/02/02p1-reader.scm

Also included below:
#!/usr/bin/env -S guile -s
!#

;; requires guile-reader >= 0.63

(use-modules
 (system reader))

(define hpos 0)
(define vpos 0)

(define (forward n)
  (set! hpos (+ hpos n)))

(define (up n)
  (set! vpos (- vpos n)))

(define (down n)
  (set! vpos (+ vpos n)))

(define fn-read (token-reader-procedure
                 (standard-token-reader 'guile-symbol-lower-case)))

(define ws-read (token-reader-procedure
                 (standard-token-reader 'character))) ;; why not 'whitespace?

(define num-read (token-reader-procedure
                  (standard-token-reader 'character))) ;; why not 'guile-number?

(define (my-token-reader chr port reader top-level-reader)
  (let* ((fn (fn-read chr port reader))
         (_ (ws-read chr port reader)) ;; read and ignore whitespace character
         (num (- (char->integer (num-read chr port reader)) 48))) ;; crude char 
code -> number
    (list fn num)))

(define my-reader
  (make-reader (list (make-token-reader '(#\a . #\z) my-token-reader)
                     (make-token-reader '(#\space #\newline #\tab) #f))))

(load "input.txt" my-reader)
(format #t "~%Result: ~a~%" (* hpos vpos))