[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PULL 13/19] checkpatch: detect doubly-encoded UTF-8
From: |
Paolo Bonzini |
Subject: |
[Qemu-devel] [PULL 13/19] checkpatch: detect doubly-encoded UTF-8 |
Date: |
Tue, 16 Jul 2019 10:11:11 +0200 |
Copy and pasting from Thunderbird's "view source" window results in double
encoding of multibyte UTF-8 sequences. The appearance of those sequences is
very peculiar, so detect it and give an error despite the (low) possibility
of false positives.
As the major offender, I am also adding the same check to my applypatch-msg
and commit-msg hooks, but this will also cause patchew to croak loudly when
this mistake happens.
Signed-off-by: Paolo Bonzini <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>
---
scripts/checkpatch.pl | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 2f81371..d24c944 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -262,6 +262,19 @@ our $UTF8 = qr{
| $NON_ASCII_UTF8
}x;
+# some readers default to ISO-8859-1 when showing email source. detect
+# when UTF-8 is incorrectly interpreted as ISO-8859-1 and reencoded back.
+# False positives are possible but very unlikely.
+our $UTF8_MOJIBAKE = qr{
+ \xC3[\x82-\x9F] \xC2[\x80-\xBF] # c2-df 80-bf
+ | \xC3\xA0 \xC2[\xA0-\xBF] \xC2[\x80-\xBF] # e0 a0-bf 80-bf
+ | \xC3[\xA1-\xAC\xAE\xAF] (?: \xC2[\x80-\xBF]){2} # e1-ec/ee/ef 80-bf
80-bf
+ | \xC3\xAD \xC2[\x80-\x9F] \xC2[\x80-\xBF] # ed 80-9f 80-bf
+ | \xC3\xB0 \xC2[\x90-\xBF] (?: \xC2[\x80-\xBF]){2} # f0 90-bf 80-bf
80-bf
+ | \xC3[\xB1-\xB3] (?: \xC2[\x80-\xBF]){3} # f1-f3 80-bf 80-bf
80-bf
+ | \xC3\xB4 \xC2[\x80-\x8F] (?: \xC2[\x80-\xBF]){2} # f4 80-b8 80-bf
80-bf
+}x;
+
# There are still some false positives, but this catches most
# common cases.
our $typeTypedefs = qr{(?x:
@@ -1506,6 +1519,9 @@ sub process {
ERROR("Invalid UTF-8, patch and commit message should
be encoded in UTF-8\n" . $hereptr);
}
+ if ($rawline =~ m/$UTF8_MOJIBAKE/) {
+ ERROR("Doubly-encoded UTF-8\n" . $herecurr);
+ }
# Check if it's the start of a commit log
# (not a header line and we haven't seen the patch filename)
if ($in_header_lines && $realfile =~ /^$/ &&
--
1.8.3.1
- [Qemu-devel] [PULL 08/19] hw/i386: turn off vmport if CONFIG_VMPORT is disabled, (continued)
- [Qemu-devel] [PULL 08/19] hw/i386: turn off vmport if CONFIG_VMPORT is disabled, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 09/19] memory: unref the memory region in simplify flatview, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 10/19] Fix broken build with WHPX enabled, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 11/19] util: merge main-loop.c and iohandler.c, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 12/19] hw/lm32/Kconfig: Milkymist One provides a USB 1.1 Controller, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 14/19] target/i386: sev: Do not unpin ram device memory region, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 15/19] hw/usb/Kconfig: Add CONFIG_USB_EHCI_PCI, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 16/19] hw/usb/Kconfig: USB_XHCI_NEC requires USB_XHCI, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 13/19] checkpatch: detect doubly-encoded UTF-8,
Paolo Bonzini <=
- [Qemu-devel] [PULL 17/19] Makefile: do not repeat $(CONFIG_SOFTMMU) in hw/Makefile.objs, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 19/19] vl: make sure char-pty message displayed by moving setbuf to the beginning, Paolo Bonzini, 2019/07/16
- [Qemu-devel] [PULL 18/19] create_config: remove $(CONFIG_SOFTMMU) hack, Paolo Bonzini, 2019/07/16
- Re: [Qemu-devel] [PULL 00/19] Bugfix/cleanup patches for 2019-07-16, no-reply, 2019/07/16
- Re: [Qemu-devel] [PULL 00/19] Bugfix/cleanup patches for 2019-07-16, Peter Maydell, 2019/07/16
- Re: [Qemu-devel] [PULL 00/19] Bugfix/cleanup patches for 2019-07-16, Peter Maydell, 2019/07/19