bug-coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

terrible Unicode shattering fold(1) command


From: jidanni
Subject: terrible Unicode shattering fold(1) command
Date: Tue, 26 Aug 2008 06:34:00 +0800

Problem 1: Here we see fold -s busy busting apart UTF-8 characters
again still.

Every third chop falls on a boundary, so is lucky.

No, I did not use `--bytes'. The result is the same as if I did anyway!

Problem 2: Also, when the critical chunk moves past the chopper blade, and we 
now start
chopping ASCII, the UTF-8 chopping subsides, but it still chops NOT at
a blank, yes, that agrees with

`--spaces'
     Break at word boundaries: the line is broken after the last blank
     before the maximum line length.  If the line contains no such
     blanks, the line is broken at the maximum line length as usual.

but don't you see **you leave no option open to respect peoples words
and not bust them apart**. "So don't use the program" you might say.
Well, the program works nicely on 99% of the lines. Just please add an
option to also not bust apart peoples words no matter what!

$ env|egrep LC_\|LANG
LC_MESSAGES=C
LC_COLLATE=C
LANG=zh_TW.UTF-8
LC_CTYPE=zh_TW.UTF-8
$ cat t3
perl -lwe '
$w=q(曾昭媛 - 婦女新知基金會|0955-327898|02-2502-8715|address@hidden|秘書長);
for(0..11){$w=~s/z/zz/;print "echo \"$w\"|fold -s|iconv|wc -lc"}'|sh
$ sh t3
      3      98
iconv: illegal input sequence at position 91
      1      91
iconv: illegal input sequence at position 92
      1      92
      3     101
iconv: illegal input sequence at position 91
      1      91
iconv: illegal input sequence at position 92
      1      92
      3     104
      3     105
      3     106
      3     107
      3     108
      3     109
$ fold --version
fold (GNU coreutils) 6.10
fmt -s does not have this problem.
fold with no -s, and no -b or with -b also busts Unicode, use this
shortened test where the blade hits the critical area. Therefore we
see -b is stuck on, like it or not.
perl -lwe '
$w=q(曾昭媛 - 婦女新知基金會|0955-327898|02-2502-8715|address@hidden|秘書長);
for(0..11){$w=~s/z/zz/;print "echo \"$w\"|fold|iconv|wc -lc"}'|sh




reply via email to

[Prev in Thread] Current Thread [Next in Thread]