[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: SMIE examples or guides
From: |
Björn Lindqvist |
Subject: |
Re: SMIE examples or guides |
Date: |
Fri, 1 Jul 2016 21:28:01 +0200 |
I managed to write the forward and backward tokenization functions:
(defun factor-smie-token (dir)
(pcase dir
('forward (forward-comment (point-max)))
('backward (forward-comment (- (point)))))
(let ((tok (buffer-substring-no-properties
(point)
(let ((syntax "w_\\\""))
(pcase dir
('forward (skip-syntax-forward syntax))
('backward (skip-syntax-backward syntax)))
(point)))))
;; Normalizes different indent starters.
(cond ((string-match factor-smie-indents-regex tok) ":")
(t tok))))
(defun factor-smie-forward-token ()
(factor-smie-token 'forward))
(defun factor-smie-backward-token ()
(factor-smie-token 'backward))
It works in 99% of the cases. But since almost any character can be
part of a token, it doesn't work perfectly. E.g te]st(ab3 would be a
perfectly valid variable name. I tried changing the syntax to
"w_()\\\"" and that fixes the tokenization but then I lose the useful
automatic indentation smie adds to opening- and closing bracket
characters. E.g I'm happy that smie indents:
[
neat
{
nice
}
(
good
)
]
It's very nice. But the following two lines are not right:
hi[there
two
They should be:
hi[there
two
2016-07-01 9:13 GMT+02:00 Stefan Monnier <monnier@iro.umontreal.ca>:
>> Many thanks! That appears to work. But there is one problem. In a
>> buffer with just this content:
>
>> [
>> \
>> ]
>
>> I get the error: (error "Bumped into unknown token")
>
> You should either give a different syntax to the \ char in the
> syntax-table, or change the :forward-token and :backward-token functions
> so they do something with \
>
>> (smie-bnf->prec2
>> '((exp (("HELLO" "HALLO" "CIAO" "SALUT") exp "BYE"))))))
>
> One way to do that is:
>
> (smie-bnf->prec2
> '((exp ("HELLO" exp "BYE")
> ("HALLO" exp "BYE")
> ("CIAO" exp "BYE")
> ("SALUT" exp "BYE"))))))
>
> Another is to change the :forward-token and :backward-token functions so
> they return the same string (e.g. "HELLO") when tokenizing any one of those.
>
>
> Stefan
--
mvh/best regards Björn Lindqvist