horrible utf-8 performace in wc

[Jan Engelhardt]

https://bugzilla.novell.com/show_bug.cgi?id=381873

Forwarding this because it is a GNU issue, not specifically a Novell one.
I reproduced this myself with the latest coreutils from git
(BTW: You might want to repack that repo, "counting objects" during the
clone was rather slow in the initial counting.)

Could it be a libiconv problem?


Glibc used: 2.8 (i586)
Coreutils: v6.11-62-g7c9c6c7


21:15 yaguchi:../rt/bin > LC_ALL=en_US time ./wc /tmp/ARC 
  3323886  30096741 537371570 /tmp/ARC

10.41user 1.86system 0:14.94elapsed 82%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (13major+177minor)pagefaults 0swaps

21:19 yaguchi:../rt/bin > LC_ALL=en_US.UTF-8 time ./wc /tmp/ARC 
  3323886  30096738 537371570 /tmp/ARC

87.21user 1.88system 1:38.89elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (2major+213minor)pagefaults 0swaps

So while ARC is definitely hot (cached) after the first run,
counting with the utf8 locale still takes up considerately more time.