Re: [Pan-users] Odd display issue

[Duncan]

Jim Henderson posted on Mon, 22 May 2017 23:26:00 +0000 as excerpted:

> On Mon, 22 May 2017 23:24:12 +0000, Jim Henderson wrote:
> 
>> I'm using a dark theme, and had noticed in earlier versions that the
>> text in the group pane seems to not consistently be displayed using the
>> color defined in the preferences - if I define white as the text color,
>> about 6 groups show up in white, but the rest show up in black.
>> 
>> Against a dark background, that makes it a little difficult to read -
>> wondering if anyone else is seeing this behaviour or knows how to fix
>> it.
>> 
>> See attached image for more info (hopefully this'll work :) )
> 
> (Apparently not)
> 
> See https://goo.gl/photos/6DiHK4CkBRdqj2wQA for the image. :)

So we have two things to discuss now, the attachment, and the groups 
color issue, both of which I /think/ I can explain... with a workaround 
for the one and I hope a fix for the other. =:^)


Let's start with the attachment.  Keep in mind that pan is primarily a 
news client, and that when it is used via gmane for this mailing list, as 
apparently both you and I do, the messages are bridged to mail via gmane, 
and while the match is close enough for most things, attachments are a 
bit of a special case due to the following...

Pan posts attachments in yEnc format, which was specifically designed for 
news and takes advantage of a not entirely standardized loophole in the 
nntp transmission and peering format to achieve far better encoding 
efficiency than the older, long standardized, formats, that work with 
both mail and news.

Both the old UUE (Unix-to-Unix Encoding) and the newer and better 
standardized MIME base64 encoding bloat binary content by roughly 33% in 
the encoding process, by encoding six original binary bits using selected 
7-bit-ASCII characters, which are then transmitted as 8-bit/byte standard 
ASCII characters.  So every eight bytes of encoded data only encodes six 
bytes of the original binary data.

They did it this way in ordered to maximize compatibility and lossless 
conversion going thru with all sorts of strange transfer and storage 
format methods and conversion between them, choosing the 64 ASCII 
characters that encode the 6 bits of binary very carefully from those 
that were cleanly represented in all storage and transfer methods known 
at the time, so the encoding could go thru multiple gateways converting 
from one format to another without loss or destruction of the binary data 
encoded within that ASCII.

yEnc improves upon that by realizing that all modern news servers and 
their peering channels are 8-bit-clean -- all they had to do was reserve 
and escape a few specific sequences, including the CRLF ASCII pair that 
terminates a line by Internet messaging standard.  Additionally, yEnc 
takes advantage of the full 1000-character (998 plus terminating CRLF 
sequence) standard line length limit, instead of wrapping at the 
otherwise standard 80 characters.  That means lower encoding overhead as 
there's fewer line-terminating CRLF sequences in the data.

As a result, yEnc achieves an encoding bloat factor of only ~5% for 
binary data, compared to the 33% or more common for the other two common 
binary encoding schemes.  Which explains why it took USENET binary groups 
by storm, even if it /wasn't/ exactly standardized.  (There have been 
some efforts to standardize it since, but with nntp being a dying 
protocol, it's unlikely to ever get thru the full process.  However, most 
if not all dedicated news providers now support it, and many news clients 
including pan do as well, because their customers and users demanded it, 
and because it was designed to work with what servers were already 
supporting in practice, beyond the standards.)

But the down side is that unlike normal MIME or UUE encoding, yEnc is 
relatively unlikely to survive mail/news and similar format-conversion 
gateways.

Which means that you can't simply attach a file in pan and send it via 
gmane to a mailing list -- even if it did survive the gateway conversion 
getting to the mailing list, for those reading the list as mail, few mail-
only clients will be able to decode yEnc at all -- even some news clients 
still lack support.

There are a couple possible workarounds, depending on where the message 
is going.  For the gmane lists-as-news special-case, probably the easiest 
for many is simply using your mail client to send the message and 
attachments directly to the list, bypassing gmane on the way in.  Mail 
clients will almost certainly default to MIME base64 encoding, which 
won't be as efficient, but should both get thru and be decodable by 
pretty much any internet message client, including both mail and news 
clients and definitely including pan.

Of course if that separate client handles news as well, either because it 
handles both mail and news, or because it only does news, but still 
defaults to MIME or UUE or lets you choose encoding format, using that 
separate client to post to the news group (via gmane in our case, but 
posting to a public group where some users are known not to have yEnc 
supporting clients works the same way), you can also post to the group 
using that separate client, and simply choose MIME or UUE encoding.

The third and perhaps most interesting workaround is to still use pan, 
but do the encoding using a separate encoding tool like uuenview from the 
uudeview package, inserting the pre-encoded result as if it were part of 
the normal text message.  Note that in this case MIME is out of the 
question, because it's not possible to setup the proper multipart message 
headers in pan to deal with MIME properly.  However, UUE is older and 
doesn't require specific headers, so works very well in this regard, and 
indeed, has been used this way from the very beginning, when people 
invented it as a method to cram binary data into a nominally text-only 
format.

And actually... before pan had built-in yEnc-based attachment 
functionality, I hacked up a couple scripts to semi-automate the 
otherwise manual encoding and attachment process. Detailing them is a 
topic for a dedicated post, but I've posted them here a number of times 
over the years, and if there's an interest, I can do so again... and 
will, as I've done before, actually use the script to attach it. =:^)


OK, so that should cover the attachments point, now on to the original 
question, the group pane group list color issue.

As it happens I /strongly/ prefer light text on dark backgrounds, what 
most people refer to as a "reversed" color theme, myself, so I knew 
exactly what you were talking about even before you posted the link to 
the image.

And as you might expect, I have a solution.  Actually, back when the 
color-coded group names feature was introduced, I had to request some 
adjustments precisely /because/ I couldn't read it the way things were!  
As it happens, that's actually why some of pan's other color prefs have 
both background and foreground color options as well, because I found it 
unworkable without that, and luckily enough for me, I've been around pan 
and this list long enough that if I complain, things usually get fixed so 
I find it workable again. =:^)

OK, so here's how it works.  There's actually two places to set group 
colors, the global one in pan preferences, colors tab, under group pane, 
where you can set the default text and background colors, and...

The per-group setting, in group preferences, where only the text/
foreground color can be set.

One point I discovered quite quickly...  If you ever expand "Other 
groups" (aka the unsubscribed list), you will definitely want a 
reasonable global/default setting, or you won't be able to read the names 
of all those unsubscribed groups, and going thru double-digit (triple-
digit?) thousands of groups to individually set them all to readability 
isn't feasible!

But for subscribed groups, and possibly for the occasional unsubscribed 
group you're looking at but haven't decided to actually subscribe yet, 
you can set individual group colors. =:^)

What I suspect has happened is that you opened and set something in group 
properties on some of the groups, without changing the default group text 
color.  Those are probably the dark ones.  Just open their group 
properties and select a different color.

Or possibly it's the reverse, you set the individual group colors and 
didn't know about the global setting.  That's actually what happened to 
me originally, because when the group colors setting was introduced, 
there /was/ no global setting, and the default text color for the per-
group setting was dark, which as you well observe, isn't particularly 
readable on a dark background.  Which was OK as long as I stayed to 
subscribed groups since I could switch the color for all of those, but 
flat-out didn't work when I went to find a group I wasn't yet subscribed 
to, since I couldn't read them and there was no way I could set them 
**ALL** individually!! So I asked that the default colors be selectable, 
background AND foreground/text, so I could set both a sane dark 
background AND a readable default foreground/text, and a bit later, it 
was. =:^)


Of course the quick way to set them all back to defaults is to
move/delete the group-preferences.xml file (with pan closed, of course), 
which contains all the group settings for any that you've changed from 
the defaults.  That of course includes group colors, but also per-group 
settings such as character-encoding, default-group-save-path, posting 
profile, etc.  So if you use different settings for those on some groups, 
just deleting the file may not be something you want to do.  Running a 
quick sed on it to delete all the color lines, while leaving the rest 
intact, should be possible, however.  Or do effectively the same thing 
using find and replace in your favorite text editor.  Just be careful not 
to break the XML formatting. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman