bug#22254: [PATCH] sed: do not elide an invalid byte in a substitution RHS

[Jim Meyering]

On Sun, Dec 27, 2015 at 10:02 PM, Jim Meyering <address@hidden> wrote:
> Just after pushing the fix for that heap-clobbering bug,
> http://debbugs.gnu.org/22127, I noticed there was another minor
> problem: sed would silently elide any invalid byte specified in a
> substitution replacement string.
>
> Here's a patch that's not quite complete (no NEWS entry, and I haven't
> yet tried to determine when it was introduced).

This one has a checkered past. I keep a decent sampling of old
versions of sed around.
Running one of the two tests that would be affected shows how
treatment of an invalid byte in a multi-byte locale has differed over
the years:

$ for i in $(env ls -dv /p/p/sed-*); do printf '%-14s: ' $i; echo a
|LC_ALL=ja_JP $i/bin/sed 's/a/b\U\xb2c/'|od -tx1|head -1; done|sed
s/.....//
sed-1.18 : 0000000 62 55 78 62 32 63 0a
sed-2.05 : 0000000 62 55 78 62 32 63 0a
sed-3.01 : 0000000 62 55 78 62 32 63 0a
sed-3.02 : 0000000 62 55 78 62 32 63 0a
sed-4.0a : 0000000 62 00 43 0a
sed-4.0b : 0000000 62 00 43 0a
sed-4.0c : 0000000 62 00 43 0a
sed-4.0d : 0000000 62 00 43 0a
sed-4.0e : 0000000 62 00 43 0a
sed-4.0.6: 0000000 62 b2 43 0a
sed-4.0.7: 0000000 62 b2 43 0a
sed-4.0.8: 0000000 62 b2 43 0a
sed-4.0.9: 0000000 62 b2 43 0a
sed-4.1  : 0000000 62 00 43 0a
sed-4.1c : 0000000 62 00 43 0a
sed-4.1d : 0000000 62 00 43 0a
sed-4.1e : 0000000 62 00 43 0a
sed-4.1f : 0000000 62 00 43 0a
sed-4.1.1: 0000000 62 00 43 0a
sed-4.1.2: 0000000 62 00 43 0a
sed-4.1.3: 0000000 62 00 43 0a
sed-4.1.4: 0000000 62 00 43 0a
sed-4.1.5: 0000000 62 00 43 0a
sed-4.2  : 0000000 62 00 43 0a
sed-4.2.1: 0000000 62 00 43 0a
sed-4.2.2: 0000000 62 00 43 0a

Now, it does what 4.0.6..4.0.9 did.