Re: [Pan-users] signature when replying to Usenet posts

[Duncan]

Dave posted on Mon, 29 Aug 2016 13:51:46 +0100 as excerpted:

> On Monday 29 August 2016 12:59:50 Dave wrote:
> 
> FYI I just checked this in Pan on Gmane and the post is badly
> misformatted.  In KMail it looks just as I posted it with all the line
> breaks intact and correct.

That originates with a conflict between format=flowed without actually 
specifying it (a kmail problem, as if it's going to try to use it, it 
should specify it), and pan's "idiosyncratic" wrapping code that tries to 
make the best of an at times messy situation and sometimes fails, but at 
least pan provides an easy wrapping toggle (by default the "w" hotkey) to 
allow quick switching between wrap modes.

Traditionally (that is pre-MIME and i10n), at the RFC "SHOULD" level, 
messages were to be line-wrapped at 72-80 characters, including 
terminating CRFL (the internet messaging RFCs define line ends as BOTH 
the CR and LF characters, in that order, and codified the SHOULD wrap as 
78 characters plus terminating CRLF), in ordered to be most easily 
readable on standard 80-char text terminals.  However, longer lines, to 
1000 characters including terminating CRLF, were /allowed/ (RFC "MUST" 
level wrapping required at 1000 chars max, 998 plus terminating CRLF).

Then came the MIME (Multi-purpose Internet Message Extensions, elsewise 
Mail for Message, but news uses the same format) RFCs/STDs, and later, a 
number of adaptations based on them such as HTML messages, format=flowed 
(the usual culprit here), the various i10n RFCs, etc.  One of the primary 
reasons for MIME was to standardize extensions to the original base 
messaging RFCs to flexibly yet within the confines of existing RFCs 
handle content not really envisioned by them originally.  This included 
handling and encoding/decoding binary content (attachments), various 
character-sets beyond 7-bit ASCII, compound messages with multiple parts 
and a way to specify how these parts are related to each other, etc.

Of particular interest here is format=flowed component of the content-
type header.  Clients that produce and honor it correctly make a 
distinction between "hard" and "soft" (soft wrap is indicated by a 
trailing space before the CRLF) line wrap when a message is marked as 
format=flowed, the idea being to allow display with non-monospace fonts 
and wrap at the width of the window instead of forcing wrap to some 
(considered by many) long outdated arbitrary monospace width such as 80 
characters.  The mechanism is combining of shorter lines (ideally within 
the 80-char RFC SHOULD-level recommendation), "flowing" them together 
where only "soft" line breaks are used, while maintaining "hard" line 
breaks to allow proper posting of tabular content in columns without 
screwing them up (at least with monospace fonts).

Unfortunately, a lot of sending clients (including it would seem, kmail 
from kde 4.x) try to do format=flowed without actually specifying it, and 
make a mess of things for receiving clients to try to unscramble as best 
they can.

Which is what we see here.  The message was sent without being designated 
format=flowed, but apparently either included terminating "soft" wrap 
sequences "SPCRLF", or was posted with full-RFC-length 998-char lines.  
(I checked the raw message file but as it was base-64 encoded all I saw 
was the encoding.  I didn't manually decode that to verify the "soft 
wraps" vs literal long lines.)

So pan decides to treat it as format=flowed despite the lack of explicit 
content-type header component stating format=flowed, and with forced-wrap 
toggled *OFF*, displays extremely long lines either where soft-wrap lines 
were combined, or where they really /were/ posted as extremely long 
lines, while properly lining up shorter tabular content with hard-wrap 
line terminations.  The tabular content can be read, but the full-line 
content is unwrapped extremely long lines.  =:^(

Meanwhile, toggling hard-wrap *ON* produces the opposite problem.  Now 
the extremely long lines are arbitrarily force-wrapped to the standard 80-
char width, making them easy to read, but the short hard-wrapped lines 
with tabular content lose their hard-wrap and are combined to fill the 80-
char width as well, making them nigh impossible to read properly as the 
tabular formatting is destroyed.


The best thing I've come up with to deal with such posts is to keep that 
wrap-toggle hotkey (again, "w" by default) handy, and make liberal use if 
it, at every paragraph break if necessary, to switch between force-
wrapped and as-posted modes.  At least then I can read a paragraph at a 
time in relative comfort, toggling modes to read the next one if 
necessary due to line vs. tabular content shifts.

Of course then the problem is that pan returns focus to the top of the 
page every time wrap is toggled, forcing me to manually scroll back down 
to where I was reading.  Which can be a chore on long posts...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman