bug#18266: handling bytes not part of the charset, and other garbage

[Paul Eggert]

Vincent Lefevre wrote:

> ypig% LC_ALL=C locale charmap
> ANSI_X3.4-1968

That may be what the 'locale' command says, but bytes with the top bit 
on are considered to be valid single-byte characters.  There are no 
encoding errors.  So, in that sense it's not strict ASCII.

> the current behavior breaks the sometimes used "grep ." solution
> to match non-empty lines.

"grep ." matches lines containing one or more characters.  Encoding 
errors are not characters, at least, not as far as plain grep is concerned.

Perhaps PCRE is different, and if libpcre worked with encoding errors we 
could simply use its way of matching them.  But there doesn't seem to be 
a safe way to do that.