[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Help-smalltalk] RBScanner and compile time constants
From: |
Holger Hans Peter Freyther |
Subject: |
Re: [Help-smalltalk] RBScanner and compile time constants |
Date: |
Sat, 21 Dec 2013 09:20:37 +0100 |
User-agent: |
Mutt/1.5.21 (2010-09-15) |
On Fri, Dec 20, 2013 at 11:11:16PM +0100, Paolo Bonzini wrote:
> Il 20/12/2013 22:21, Holger Hans Peter Freyther ha scritto:
> > Now my question is if the optimized token should recursively scan the
> > literals? Or should the RBScanner count $( and $) to know that it needs
> > to read one more item?
>
> Keeping track of the number of parentheses, and exiting scanLiteralArray
> when it matches the count at entry, sounds like the easiest solution.
>
> Only #scanLiteralArray and #scanSpecialCharacter deal with parentheses.
Before reading your mail I came up with that. I can change it to add
counting of currentCharacter==$( and currentCharacter==$) in RBScanner>>#step
instead?
With the below diff I can tokenize/parse #(##(1/2)) the only issue is that
compiling it doesn't work...
Works:
res := STEvaluationDriver new
parseSmalltalkStream: '##(1/2)' readStream
with: GSTFileInParser.
self assert: res equals: 1/2.
Doesn't:
res := STEvaluationDriver new
parseSmalltalkStream: '#(##(1/2))' readStream
with: GSTFileInParser.
res inspect.
STInST.RBOptimizedToken(Object)>>doesNotUnderstand: #realValue
(SysExcept.st:1408)
optimized [] in STInST.RBLiteralToken>>realValue (RBToken.st:227)
Array(ArrayedCollection)>>collect: (ArrayColl.st:307)
STInST.RBLiteralToken>>realValue (RBToken.st:227)
STInST.RBLiteralNode>>value (RBParseNodes.st:2616)
STInST.STCompiler>>acceptLiteralNode: (STCompiler.st:626)
STInST.RBLiteralNode>>acceptVisitor: (RBParseNodes.st:2654)
STInST.STCompiler>>acceptReturnNode: (STCompiler.st:896)
STInST.RBReturnNode>>acceptVisitor: (RBParseNodes.st:1594)
optimized [] in STInST.STCompiler>>compileStatements: (STCompiler.st:570)
Array(SequenceableCollection)>>keysAndValuesDo: (SeqCollect.st:886)
STInST.STCompiler>>compileStatements: (STCompiler.st:571)
STInST.STCompiler>>acceptSequenceNode: (STCompiler.st:447)
STInST.RBSequenceNode>>acceptVisitor: (RBParseNodes.st:981)
STInST.STCompiler(STInST.RBProgramNodeVisitor)>>visitNode: (RBParseNodes.st:49)
STInST.STCompiler class>>compile:asMethodOf:classified:parser:environment:
(STCompiler.st:161)
STInST.STEvaluationDriver>>evaluate: (STEvaluationDriver.st:209)
diff --git a/packages/stinst/parser/RBParser.st
b/packages/stinst/parser/RBParser.st
index b953762..3314ff9 100644
--- a/packages/stinst/parser/RBParser.st
+++ b/packages/stinst/parser/RBParser.st
@@ -1215,16 +1215,20 @@ Stream subclass: RBScanner [
scanLiteralArray [
<category: 'private-scanning'>
- | arrayStream start |
+ | arrayStream start optimizedNodes |
arrayStream := WriteStream on: (Array new: 10).
self step.
start := tokenStart.
+ optimizedNodes := 0.
[self stripSeparators.
tokenStart := stream position.
- currentCharacter == $)]
+ currentCharacter == $) and: [optimizedNodes = 0]]
whileFalse:
- [arrayStream nextPut: self scanLiteralArrayParts.
+ [ |res|
+ res := arrayStream nextPut: self scanLiteralArrayParts.
+ res isOptimized ifTrue: [optimizedNodes := optimizedNodes +
1].
+ res isSpecial ifTrue: [optimizedNodes := optimizedNodes -
1].
buffer reset].
self step.
^RBLiteralToken
@@ -1254,6 +1258,8 @@ Stream subclass: RBScanner [
currentCharacter == $$ ifTrue: [^self scanLiteralCharacter].
currentCharacter == $( ifTrue: [^self scanLiteralArray].
currentCharacter == $[ ifTrue: [^self scanByteArray].
+ "In the case of RBOptimizedToken we scan for $) as well"
+ currentCharacter == $) ifTrue: [^self scanSpecialCharacter].
^self scannerError: 'Unknown character in literal array'
]