[Pan-users] Re: Re: How to upload attachments?

[Duncan]

Kevin Brannen posted <address@hidden>, excerpted below,  on
Wed, 18 Jan 2006 21:03:01 -0600:

>  I don't know what RFCs (if any) apply to making attachments, but if 
> someone can enlighten me, I'll poke around and see if something
> happens.

PAN is MIME based.  Take a look at the headers of a post, and see.  The
MIME RFCs, including encoding, are 2045-2049.  Currently, PAN sets
Content-type: Text/plain, Content-transfer-encoding: 7bit.

If it's ONLY an attachment, no text, the normal MIME
Content-transfer-encoding is Base64 (name due to using 64 characters for
encoding, thus, each character encodes six bits of data). There's also
Quoted-printable, which is mostly plain text but with "=" used as an
escape character, normally followed by a two-hexit code, therefore
allowing any 8-bit byte code to be encoded as a three char =FF sequence.

Obvously, base64 is FAR more efficient at encoding binary data than
quoted-printable, but quoted-printable is designed to safely encode
primarily text data that might have a few non-messaging-gateway-safe data
bytes, while keeping it basically readable.

There's 7-bit identity content-transfer-encoding, which is what PAN
normally uses, and 8-bit identity coding, to be used ONLY where 8-bit
encoding is known to be safe.

If it's text with an attachment, both, then it would be
Content-type: multi-part/mixed, there would be a header giving the
boundary-code, and then each part would be a sort of mini-message of its
own, with its own content-type header, etc, and the boundary-code marking
off the boundary between parts.

There's also a partial-part, allowing a message to be split over several
segments, but I don't remember the official content-type for it.

There's no official standard for UUEncode based encoding.  It's an earlier
type than MIME, with XXEncode, now long outdated, previous to that. 
Because there's no official definitive UUE standard, implementations
differ somewhat, and are occasionally incompatible.  Due to this and
because the overhead is roughly the same as MIME/Base64, few MIME based
clients implement UUE based attachments.  The only case where it would be
implemented would be if there was a concern about backward compatibility
with something not yet MIME compliant.

yEnc is the newest and most efficient widely used coding method, but can
ONLY safely be used for news, NOT MAIL. Where both UUE and Base64 have a
30-something percent overhead, encoding 6-bits to the byte, yEnc's
overhead is only about 5%, MUCH better.  It accomplishes this by using
full 8-bit encoding, doing a bit-shift and escaping ONLY the characters
necessary for protocol signalling (CRLF, for instance).  It can get away
with this because the main news system is known to be basically 8-bit
safe, with the exception of a few characters, which are escaped.  However,
because the same does NOT apply to some portions of the mail system, yEnc
cannot be safely used for mail encoding.  It is, as I said, news-only. 
yEnc is also new enough that many clients don't handle it, yet.  MSOE is
the most widely used of these.  yEnc is not yet an official standards
track RFC (or wasn't last I knew, tho my knowledge is a bit dated, so it
might be, now), and the MS position was that they weren't going to
implement it until/unless it went standards track.  However, because it is
SO much more efficient than the other encoding methods, it has become
INCREDIBLY popular in the binary groups.  MSOE and other incompatible
client users are told to quit bitching and get a decent news client, as
it's the posters who choose their format, and because of the lower
overhead, they can either post more in the same time, or get their posting
done in a shorter time, using yEnc, and that's exactly what they do.

Besides the MIME RFCs, there's Unix oriented UUE and MIME/Base64
documentation in the UUDeview/UUEnview documentation.  There's also a
C based library, with a 31-page PDF worth of documentation.  Also see the
standard IEEE/The Open Group UUEncode manpage.  It has a **VERY** helpful
table and description of both UUEncode and Base64.

For yEnc, http://www.yenc.org is the definitive site.  Also see
http://yencode.org for a Unix based GPL encoder/decoder implementation. 
(Checking yenc.org, it looks like there IS a standards-track effort, now,
in fact expanding on yEnc.)

For PAN attachments, you'll certainly want to support yEnc, as that's
what virtually all the binary groups are using, now.  You'll also want
Base64 unless you are only implementing one form, as that will give
posters the choice of being MSOE compatible. UUE would be the low
priority, only if you want to maximize backward compatibility as a choice,
as by this point, everybody that's at all into binary has at LEAST
MIME/base64 support in their client, and anybody serious about binaries
has yEnc support.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html