[Pan-users] Re: Re: How to upload attachments?

[Duncan]

Duncan posted <address@hidden>, excerpted below, 
on Sun, 15 Jan 2006 10:19:24 -0700:

> Kevin Brannen posted 
> <address@hidden>, excerpted
> below,  on Thu, 12 Jan 2006 21:50:48 -0600:
> 
>>>2) Similarly, use the external text editor function.  It should be
>>>possible, and I've suggested it before but don't know that anyone has
>>>actually done it (if they have, they've not posted it), to script a
>>>solution that does the attachment, then saves the combined message and
>>>attachment, and set that script as your "external editor" in PAN's
>>>preferences dialog, such that when PAN gets control back from the
>>>"external editor", the attachment is now part of the message.  I /do/
>>>know someone has used a very similar technique to allow gpg-signing of
>>>messages, as that's where I got the idea.  (Unfortunately, I'm not sure
>>>that script was ever posted, tho someone commented they had come up
>>>with one.)
>>>  
>> Hmm, interesting idea.  I might have to try to figure that out.
> 
> OK, I've been thinking about this a bit, and might see what I can come
> up with later this week.

...  And so I did.

> The alternative that can actually work, therefore, is using something like
> uuenview (uudeview) to do the actual file encoding, but having PAN handle
> the posting, since only it has the necessary header information.  This of
> course means that the glue-code script will be SERIOUSLY more complicated,
> since it will have to take the actual encoded file and insert it into the 
> message correctly, adding appropriate attachment begin and end flags as
> appropriate.

Well, I'm not sure how other news clients will see  them, but the
attachment glue code didn't turn out to be so bad after all, for
text/uuencode/yenc.  I did indeed use uuenview.  It can also do base64 and
quoted-printable MIME, but that would involve some hand-editing of the
headers post-insert, and possibly some PAN code tweaking, to get right, so
I left that well enough alone.

UUDeview/UUEnview is available at the following URL.  The script uses
uuenview only:  http://www.fpx.de/fp/Software/UUDeview/

Setup the script to be invoked from PAN as pan-attach %t -- the %t will be
replaced by the filename when the external editor button is pushed.  The
setting is in Preferences, on the mail and apps tab, external editor.

Text "attachment" is basically simply identity encoding, aka "rot-52". <g>
The "attachment is simply added to the message and extends it.  One could
make it a separate part with a bit of header editing, but as I said, I'm
staying away from that, because to do it right would involve getting into
PAN's guts and I've no wish or skill to get that far into it.

UUEncoding is done mostly properly, AFAIK, but it's very uncommon for a
MIME compliant client to do UUE in MIME, as Base64 is much better
defined and part of the spec.  PAN is a MIME compliant client, that
includes the MIME 1.0 and content text/plain headers in its posting, and
an add-on script won't change that, so some clients won't see the
attachment because it says it's MIME plain-text, and they'll expect it to
be just that and not /look/ for an attachment.

Note that in my testing, I discovered that PAN will see the attachment
just fine, and change the headers it DISPLAYS to content multi-part/mixed.
So, yes, it sees the attachments just fine, and corrects the headers
accordingly.  I'm not sure if it corrects them on disk (in cache) or not,
only displaying the fixed ones or saving the fixed ones to disk as well. 
I'll look into that, but not today...  Anyway, just a caution not to trust
what PAN tells you about the headers  when you download your own posts. 
KNode sees them rather differently, and can't see the attachments (not
that it makes a good binary news client anyway as it won't even display
them -- it makes you open them to see them -- but that's how I found PAN
was "fixing" the headers.  I haven't figured out how to make klibido even
see the posts, yet -- it supposedly only sees (or shows overviews for)
binaries, but I haven't figured out how  to get them to show up.  Maybe  
it's trusting the MIME header saying text-plain?  I may have to take a
look at the code and see.  (I can usually read C/C++, if it's commented
and modular enough, just can't write it.)

Yenc is pretty much likewise.  I'm not sure if the main yenc spec still
calls for yenc in the subject header, as I haven't looked at it for a
couple years and it wasn't stable yet back when I DID read the spec. If
it's still a part of the spec, a LOT of modern poster-bots ignore it,
including Power-Post, I believe. Anyway, the script only gets the body to
deal with, so if you want to put yenc in the subject, you do that manually.

The MIME headers saying plain-text will of course be a problem with yEnc
as well, but I expect a bit less of one, as yenc is pretty new and not
fully stable in implementation yet (as I said, I'm not sure  if the spec
is yet or not, but it wasn't a couple years ago when I last looked at it).
Thus, yEnc decoders are likely to be rather more tolerant of minor
aberrations from accepted norms, because it's not set in stone like MIME
has been for many years already.  KLibido supposedly does yEnc but I can't
get it to detect them as binaries and therefore display the overviews so I
can download the bodies, so what it's doing remains a mystery.  PAN sees
and displays them fine, and KNode doesn't do yEnc, so wouldn't be expected
to see them in any case.

> I'm going to try to keep this first implementation non-interactive and as
> simple as possible, with a main goal of actually getting a solution out
> there, even if it's very crude. Picking the file to attach therefore
> becomes an issue, as well as telling the "editor" which encoding type to
> use.  yEnc must be an option here or it's not worth my trouble, yet due to
> the fact that many clients don't support it yet, uuencoding and/or
> MIME/base64 should be optiions as well.

The attachment typing has already been covered, but when I said a crude
interface, that's EXACTLY what it has.  I'm not even sure it's better than
PAN's own NO attachment posting at this point, in terms of functionality
that can really be used, but it works in a pinch and for testing, which
for a few hours' hacking isn't too bad, I suppose, particularly when that
was spent mostly getting the egrep/sed/cut stuff just right -- I spent FAR
longer on that than a person really familiar with it would/should have
done.  Still, I've waited for someone who can hack it out in half an hour
to do so... for several years now... and just decided to see what I COULD
do, even if I wasn't going to be efficient at it.  It works... as a very
crude hack, which is farther than it was before.  Maybe my posting it will
stimulate someone with some real skill to take it and run with it -- or
better yet, with PAN itself.  If not, I'll probably do a couple more
iterations with it, if not before, then when I get into Python, which
could be anything from a couple weeks to a couple years away...

> I /said/ it was crude, but here's a method that should work -- make the
> glue-script recognize a couple key variables -- and their escapes.  So,
> something like "$$attach=/path/to/the/file/to/attach.jpg$$"
> (without the quotes) delimited by the $$ signs, all alone on its own line,
> will be detected by the script, with it deleted and the file it points to
> attached to the end of the message. Likewise, "$$encoding=yenc$$" on its
> own line will be detected, honored, and removed.  That may do it, since
> few of the other options would make much sense in that environment. 

Since I'm doing it in BASH and it already makes heavy use of the $ sign, I
found that didn't work too well.  I switched to the % sign, specifically:

%%ENCODE=xxxx%% for the encoding type.  yenc's the default, so need not
be specified.  All encodings are specified as four-letter lower case, so
those xxxx's can be replaced with text|uuen|yenc.  The script is looking
specifically for UPPER-CASE ENCODING, lower case type, four letters of
type, with the equals in the center, and double-percent signs on either
side.  It *MUST* be on its own line, as well.  The idea is to make it very
unlikely for the sequence to occur in a regular conversation and then get
accidentally replaced, should an attachment be added (and thus the script
invoked) as well.

I DID say it was crude!

Similarly to specify the file: %%ATFILE=/path/to/filename%%, on its own
line.  If there are percents in the path, it should still work, as long as
the double-percent is at each end.  DON'T FORGET THE TERMINATING %% AS I
DID IN TESTING A COUPLE TIMES, THEN WONDER WHAT'S WRONG!!  I haven't yet
tried it with spaces in the name, but it /should/ work in theory, tho it
might require a slight adjustment of the script.  That's because the test
is line-based, then looks for the %% at each end and the ATFILE= sequence
after the opening %%, so a space /shouldn't/ cause issues -- but I haven't
tested it, and I didn't worry about quote-protecting the thing, either, so
it might not.

> The problem of course is having to specify the path and file in long form
> without tab completion or a file selection dialog.  

Exactly.  I tried a ~/ style path, and it didn't work.  I didn't try
relative paths as I haven't gotten that far in testing yet and didn't know
for sure what it was running out of, so didn't worry about it.  It DOES
work (here) with absolute paths, of the form
/home/username/subdir/filename, however.

> However, it'll be omething we don't have now, and improvements from
> that can be done later, if desired.  I (or someone else) can then either
> change the script to use xdialog or the like for interactivity, or
> rewrite it in python, using its more advanced scripting and library
> techniques to handle the GUI part.


> License will be identical to PAN's (GPL).

There's a copyright and GPL 2 license statement in the file.  It's
designed to be used with PAN and says it's licensed identically, as well,
license, the COPYING file, to be attached to a subsequent post.

A couple hints about use:  

* The script will use (and eat) the first properly formatted ATFILE and
ENCODE lines it fines in the body, ignoring any further such lines. 
Therefore, while it only attaches a single file at a time, you can set up
several file paths (with optional encoding for each), then click the
external editor button repeatedly to eat them up, one at a time, attaching
each one to the end of the previous one, with a couple blank lines for
spacing in between.  Of course, if you have less ENCODING entries than you
do ATFILE entries, it will default to yenc after using up the ones you
specified.

* I haven't tried this yet with an X based editor, but I setup an
environmental variable, PAN_EDITOR.  If you have that set to a valid X
based editor, and exported, pan-attach (the script name) should exec it,
passing the $BODYFILE as the only parameter, if it doesn't find a valid
ATFILE line. That means you can still use your normal external editor to
edit the file, if desired, as long as there aren't any valid ATFILE
entries for pan-attach to glob onto.  To make it work correctly since PAN
waits for the application to close then grabs the file back, I've used
bash's exec command to exec the editor as the same process.  Do NOT use a
standard CLI editor, without calling an xterm to load it in -- I made that
mistake in testing and had to kill PAN to get things working again. 
That's also why I'm not usng the standard $EDITOR variable.

OK, now we'll see what GMANE takes for attachments.  I'm going to try UUE
first, as that should protect the formatting.  If  it doesn't work, I'll
attach as text, but be aware that the line-wrap may get messed up.  If you
are using gmane and viewing the group/list with PAN, the UUE attachment
should come thru if gmane allows it. otherwise it'll show up as an
extension to a text post, below my sig. If you are using a regular mailer,
how it handles the attachments will probably be the deciding factor in
whether you see it or not (assuming gmane takes it).

Again, the COPYING license file will follow as soon as I see it has
posted, posted the same way.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html



begin 740 pan-attach
M(R$O8FEN+V)A<address@hidden(R!#;W!Y<FEG:'0@,C`P-B!*;VAN($1U;F-A;@HC($1I
M<W1R:6)U=&5D('5N9&5R('1H92!T97)M<R!O9B!T:&address@hidden($=E;F5R86P@
M4'5B;&EC($QI8V5N<address@hidden(N"B,@5&AI<R!S8W)I<'0@:7,@9&5S:6=N960@
M=&address@hidden:R!W:71H(%!!3B!A;address@hidden&ES=')address@hidden(@=&AE"B,@
M<V%M92!L:6-E;G-E(&-O;F1I=&EO;G,N("!H='1P.B\O<&%N+G)E8F5L8F%S
M92YC;VTO(&]R(&1I<F5C=&QY(&9R;address@hidden&AE"B,@1G)E92!3;V9T=V%R92!&
M;W5N9&%T:6EO;B!H='1P.B\O=W=W+F=N=2YO<F<O;&EC96YS97,O9W!L+FAT
M;6PN("!))VQL"B,@<&]S="!A(&-O<'EI;F<@9FEL92!W:&5N($D@<'5T('1H
M:7,@=7`@;VX@;address@hidden('!A9V4L('address@hidden;W5N8V4*(R!I="!T;R!T
M:&address@hidden(%5315)3(&UA:6QI;F<@;&ES="X*"DY!344](G!A;BUA='address@hidden
M"E9%4E-)3TX](C(P,#4N,#$N,3<N,B(*"B,@4V-R:7!T(&1E<VEG;F5D('1O
M(&)E(&address@hidden,@4$%.)W,@97AT97)address@hidden&]R+"!T;R!B92!U
M<V5D('1O"B,@871T86-H(&9I;&address@hidden@<F5Q=6ER97,@=75E;G9I97<L
M('!A<G0@;address@hidden&AE('5U9&5V:65W('!A8VMA9V4L"B,@879A:6QA8FQE(&9R
M;VT@(&AT='`Z+R]W=W<N9G!X+F1E+V9P+U-O9G1W87)E+U551&5V:65W+R`N
M"@HC(%1H92!S8W)I<'0@('1A:V5S(&$@<VEN9VQE('!A<F%M971E<BP@;F]R
M;6%L;'address@hidden&AE(&9I;&address@hidden&%I;FEN9PHC('1H92!B;V1Y('1E>'address@hidden,@
M<&%S<V5D(&9R;address@hidden@H*0D]$649)3$4])#$*"B,@5VET:&EN('1H870@
address@hidden&AE('-C<FEP="!W:6QL(&QO;address@hidden('1W;R!M86=I8R!S=')I
M;F=S+`HC(&]F('1H92!F;W)M("0D/%9!4CX]/'9A;'5E/B0D+"!W:&EC:"!-
M55-4(&5A8V@@8F4@;address@hidden:6YE(&]F"B,@:70G<R!O=VXN"@HC("4E14Y#
M3T1%/3QT>7!E/B4E"B,)=VAE<address@hidden<&5=(&ES(&address@hidden;BP@
M>65N8R`H9&5F875L="DL(&]R('1E>'0N"B,)*'5U96YV:65W(&%L<address@hidden
M;V1E<R!T;address@hidden('%U;W1E9"UP<FEN=&%B;&4L"B,):&]W979E
M<BP@<V-R:7!T:6YG('1H96ER(&AE861E<G,@=V]U;&0@<F5Q=6ER92!A9&1I
M=&EI;VYA;`HC"6AE;'address@hidden)O;address@hidden&AE($9)4E-4(&EN<W1A;F-E
M(&ES('1A:V5N+"!O=&AE<G,@87)E(&EG;F]R960N"B,*(R`E)4%41DE,13T\
M+W!A=&@O9FEL93XE)0HC"4]N92!F:6QE(&%T(&address@hidden&address@hidden(&YO=RX@
M($%G86EN+"!&25)35"!I;G-T86YC92X*(PHC(R,C(R,C(R,C(R,C(R,C(R,C
M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C
M(R,C(R,C(R,C(R,C(R,C(PHC(%1H:7,@<V5C=&EO;B!S971S('1H92!E;F-O
M9&EN9R!P87)A;65T97(L("1[14Y#3T1%05-]+"!T;R!U?'E\="P*(R!A;F0@
M9&5L971E<R!T:&address@hidden<F5S<&]N9&EN9R!M86=I8R!S=')I;F<@;&EN92!F
M<F]M("1[0D]$649)address@hidden(R!'<F%B("4E14Y#3T1%/71E>'1\=75E;GQY
M96YC)24@;6%G:6,@<W1R:6YG+"!A;&]N9R!W:71H(&ET<R!L:6YE;F\N"D5.
M0T]$14%3/address@hidden>&TQ("4E14Y#3T1%/5PH=&5X=%Q\=75E;EQ\
M>65N8UPI)24@)'M"3T191DE,17TI"FEF(%L@)'M%3D-/address@hidden@=&AE
M;@H)(R!792!R96-O9VYI>F5D(&$@;6%G:6,@96YC;V1E('-T<FEN9R!L:6YE
M"@DC($1E;&5T92!T:&%T(&QI;F5N;RX*"7-E9"`M:2`D*&5C:&\@)'M%3D-/
M1$5!4WU\8W5T("UD.B`M9C$I9"`D>T)/1%E&24Q%?0H)(R!4<FEM('1H92!L
M:6YE;F\N"@E%3D-/1$5!4STD*&5C:&\@)'M%3D-/1$5!4WU\8W5T("UD.B`M
M9C(I"@DC(%1R:address@hidden&AE('1Y<&address@hidden&\@<VEN9VQE+6-H87(*"45.0T]$14%3
M/21[14Y#3T1%05,Z.3HQ?0IE;'-E"@DC(&1E9F%U;'1E9`H)14Y#3T1%05,]
M>0IF:0H*"B,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C
M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C"B,@
M5&AI<R!S96-T:6]N(&=R86)S('1H92!F:6QE('1O(&%T=&%C:"P@)'M!5$9)
M3$5]+"!A;address@hidden&5L971E<R!T:&4*(R!C;W)R97-P;VYD:6YG(&UA9VEC('-T
M<FEN9R!L:6YE(&9R;VT@)'M"3T191DE,17TN"@HC($=R86(@)25!5$9)3$4]
M+W!A=&@O9FEL96YA;64E)2!M86=I8R!S=')I;F<@86YD(&QI;F5N;RX*051&
M24Q%/address@hidden>&TQ("4E051&24Q%/2XJ)24@)'M"3T191DE,17TI
M"@HC($EF(&YO(&9I;&address@hidden&address@hidden"!L970G<R!A<W-U;address@hidden&AE>2!J
M=7-T('=A;G1E9"!T;R!E9&ET('1H92!F:6QE+BXN"B,@86YD(&AO<&address@hidden&AE
M>2!H879E(&$@<V%N92`D141)5$]2('-E="X*:address@hidden("1[051&24Q%?2!=
M.R!T:&5N"@EI9B!;("UX("1[4$%.7T5$251/address@hidden@=&AE;@H)"65X96,@
M)'M004Y?141)5$]2?2`D>T)/1%E&24Q%?0H)96QS90H)"65C:&\@(D-O=6QD
M;B=T(&9I;address@hidden:6QE('1O(&%T=&%C:"$@($EF('EO=2!W86YT('1O('!A
M<W,B"@D)96-H;R`B=&AR;W5G:"!T;R!A;B!E>'1E<FYA;"!E9&ET;W(@;V8@
M>6]U<B!C:&]I8V4L(&5X<&]R="(*"0EE8VAO(")T:&address@hidden"1004Y?141)5$]2
M('9A<FEA8FQE+"!A;F0@<&%N+6%T=&%C:"!W:6QL(&-A;&P@:70B"@D)96-H
M;R`B87,@7"1004Y?141)5$]2(%PD0D]$649)3$4B"@D)97AI="`Q"@EF:0IF
M:0H*(R!792!R96-O9VYI>F5D(&$@;6%G:6,@871T86-H+69I;&4@<W1R:6YG
M(&QI;F4L(&)U="!D;VXG="!K;F]W(&EF(&ET)W,@=F%L:60@>address@hidden(%-T
M:6QL+"!D96QE=&address@hidden&AA="!L:6YE;F\N(&9R;VT@)'M"3T191DE,17T*<V5D
M("UI("0H96-H;R`D>T%41DE,17U\8W5T("UD.B`M9C$I9"`D>T)/1%E&24Q%
M?0HC(%1R:address@hidden&AE(&QI;F5N;RX*051&24Q%/20H96-H;R`D>T%41DE,17U\
M8W5T("UD.B`M9C(I"B,@5')I;2!T;R!T:&4@<&%T:"`F(&9I;&4*051&24Q%
M/21[051&24Q%.CE]"D%41DE,13TD>T%41DE,124E)25]"@HC27,@)'M!5$9)
M3$5](&%C='5A;&QY(')E861A8FQE/address@hidden@;F]T+"!E<G)O<B!O=70N"FEF
M(%L@(2`M<B`D>T%41DE,address@hidden@=&AE;@H)96-H;R`B5&AE(&9I;&4@>6]U
M('-A:address@hidden&address@hidden"(*"65C:&\@(B1[051&24Q%?2(*"65C:&\@(G=A
M<VXG="!R96%D86)L92$B"@EE>&ET(#(*9FD*"B,C(R,C(R,C(R,C(R,C(R,C
M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C
M(R,C(R,C(R,C(R,C(R,C(R,C"B,@3TLL('=E(&MN;W<@=&AE(&%T=&%C:"!F
M:6QE(&ES(')E861A8FQE+"!A;address@hidden@:VYO=R!W:&%T(&5N8V]D:6YG('1O
M('5S92P@<V\N+BX*96-H;R`^/B`D>T)/1%E&24Q%?0IE8VAO(#X^("1[0D]$
M649)3$5]"G5U96YV:65W("TD>T5.0T]$14%3?2`D>T%41DE,17T@/CX@)'M"
M3T191DE,17T*"B,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C
M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C
M"B,@9&5B=6=G:6YG(&%I9',*(VMO;G-O;&address@hidden;V-L;W-E("TM;F]H:7-T
C("UE(&5C:&\@)`HC97AE8R!K=W)I=&4@)$)/1%E&24Q%"@H`
`
end