Re: [emacs-bidi] testing bidi.el with the pgba tests

[Alex Schroeder]

"Eli Zaretskii" <address@hidden> writes:

> Please do.  There are also a few interesting test cases here:
>
>   http://crl.NMSU.Edu/~mleisher/ucdata.html

Hm, the results are a bit disappointing, to be honest.  Some of it
should have been obvious -- I don't do mirroring of selected glyphs
(rule L4).  Does Emacs have the required data somewhere?

Punctuation also seems to be a killer.  I didn't implement Ehud's
idea, yet, because I'm still not really sure where it fits in.

Alex.


VISUAL TO LOGICAL

Column 1 has the results from the logical-to-visual transformation
from the Unicode Reference Code column.  Column 2 has the result of
running bidi-visual-to-logical-paragraph in a left-to-right context on
it.  Column 3 has the result of running the same function in a
right-to-left context on it.

An X just before the bidi-visual-to-logical-paragraph indicates that
neither column matched the source string provided as input to the
Unicode Reference Code.

Sources of differences are:

1. My code doesn't do mirroring, for example of parenthesis characters
   in number 4.

2. It seems that my code didn't invert 5-1 and 5/1 in number 15.  I
   will have to investigate.

car is RAC EHT in arabic                car is THE CAR in arabic                
in arabic THE CAR car is
HSILGNE NI the car SI RAC               IN ENGLISH the car CAR IS               
CAR IS the car IN ENGLISH
he said "KO ,456 ,123 SI TI"          X he said "IT IS ,123 456, OK"            
"IT IS ,123 456, OK" he said
he said "KO ,(456 ,123) SI TI"        X he said "IT IS ),123 456(, OK"          
"IT IS ),123 456(, OK" he said
he said "KO ,123,456 SI TI"             he said "IT IS 123,456, OK"             
"IT IS 123,456, OK" he said
he said "KO , (123,456) SI TI"        X he said "IT IS )123,456( , OK"          
"IT IS )123,456( , OK" he said
"it is 123, 456, ok" DIAS EH            "it is 123, 456, ok" HE SAID            
HE SAID "it is 123, 456, ok"
<123H/>shalom<123H>                   X <123H/>shalom<123H>                     
>Hshalom<123>/H123<
<h123>MALAAS</h123>                     <h123>SAALAM</h123>                     
>h123/<SAALAM>h123<
NAR DNA "!it is a car" DIAS EH          AND RAN "!it is a car" HE SAID          
HE SAID "it is a car!" AND RAN
NAR DNA "it is a car!x" DIAS EH         AND RAN "it is a car!x" HE SAID         
HE SAID "it is a car!x" AND RAN
DLOC SI SUISLEC -2                      -2 CELSIUS IS COLD                      
-2 CELSIUS IS COLD
EGNAHC -10%                             -10% CHANGE                             
-10% CHANGE
5+1 5/1 5-1 5*1 EVLOS                 X 5+1 5/1 5-1 5*1 SOLVE                   
SOLVE 1*5 5-1 5/1 5+1
5..2.5 SI EGNAR EHT                   X 5..2.5 THE RANGE IS                     
THE RANGE IS 5..2.5
he said "RAC A SI TI!"                  he said "IT IS A CAR!"                  
"!IT IS A CAR" he said
he said "X!RAC A SI TI"                 he said "IT IS A CAR!X"                 
"IT IS A CAR!X" he said
abc (TSET)                            X abc (TEST)                              
)TEST( abc
abc (TSET)                              abc (TEST)                              
)TEST( abc
TSET address@hidden                                TEST address@hidden          
                address@hidden TEST
abc ECNO 23 TSET                        abc TEST 23 ONCE                        
TEST 23 ONCE abc
abc ECNO 23%%%~~~ TSET                  abc TEST ~~~23%%% ONCE                  
TEST ~~~23%%% ONCE abc
abc ECNO abc ~~~23%%% TSET              abc ONCE abc ~~~23%%% TEST              
TEST abc ~~~23%%% ONCE abc
ECNO address@hidden@cde TSET                    ONCE address@hidden@cde TEST    
                TEST address@hidden@cde ONCE
ECNO abc 23 cde TSET                    ONCE abc 23 cde TEST                    
TEST abc 23 cde ONCE
cde ECNO abc 23 TSET                    cde ONCE abc 23 TEST                    
TEST abc 23 ONCE cde
Z a 2X                                  Z a 2X                                  
Xa 2 Z



LOGICAL TO VISUAL

Column 1 is from the test file, column 2 is the result of running
bidi-logical-to-visual-paragraph on that line, with automatic context
detection, ie. the first strong character determines the context.

An X just before the bidi-logical-to-visual-paragraph indicates that
the result doesn't match the output produced by the Unicode Reference
Code.

Sources of differences are:

1. My code doesn't do mirroring, for example of parenthesis characters
   in number 4.

2. My code doesn't switch 1+5 to 5+1 in number 14.  The Unicode
   Reference Code seems to produce 5+1, all other implementations
   listed keep 1+5.

3. There seems to be a bug in my code for number 15.  Im not sure what
   went wrong in this case, I'll have to look at it again.

# A test with capital letters treated as RTL
car is THE CAR in arabic                          car is RAC EHT in arabic
CAR IS the car IN ENGLISH                         HSILGNE NI the car SI RAC
he said "IT IS 123, 456, OK"                      he said "KO ,456 ,123 SI TI"
he said "IT IS (123, 456), OK"                  X he said "KO ,)456 ,123( SI TI"
he said "IT IS 123,456, OK"                       he said "KO ,123,456 SI TI"
he said "IT IS (123,456), OK"                   X he said "KO ,)123,456( SI TI"
HE SAID "it is 123, 456, ok"                      "it is 123, 456, ok" DIAS EH
<H123>shalom</H123>                             X >123H/<shalom>123H<
<h123>SAALAM</h123>                               <h123>MALAAS</h123>
HE SAID "it is a car!" AND RAN                    NAR DNA "!it is a car" DIAS EH
HE SAID "it is a car!x" AND RAN                   NAR DNA "it is a car!x" DIAS 
EH
-2 CELSIUS IS COLD                                DLOC SI SUISLEC -2
-10% CHANGE                                       EGNAHC -10%
SOLVE 1*5 1-5 1/5 1+5                           X 1+5 1/5 1-5 5*1 EVLOS
THE RANGE IS 2.5..5                             X .5.2.5 SI EGNAR EHT
#
#
# Adapted from one of the FriBidi test files.
#
he said "IT IS A CAR!"                            he said "RAC A SI TI!"
he said "IT IS A CAR!X"                           he said "X!RAC A SI TI"
(TEST) abc                                      X abc )TSET(
abc (TEST)                                        abc (TSET)
 address@hidden TEST                                         TSET address@hidden
TEST 23 ONCE abc                                  abc ECNO 23 TSET
TEST ~~~23%%% ONCE abc                            abc ECNO 23%%%~~~ TSET
TEST abc ~~~23%%% ONCE abc                        abc ECNO abc ~~~23%%% TSET
TEST address@hidden@cde ONCE                              ECNO 
address@hidden@cde TSET
TEST abc 23 cde ONCE                              ECNO abc 23 cde TSET
TEST abc 23 ONCE cde                              cde ECNO abc 23 TSET
Xa 2 Z                                            Z a 2X

-- 
http://www.emacswiki.org/