[Pan-users] In the new file-upload logic, we n eed to change how Pan apparently slices the binary data …

[SciFi]

Hi,

I'm contemplating the file-upload feature being put into Pan.

I'm speaking for use in the way
how massive binary files are being uploaded
to the world-wide Usenet,
their actual expected formats & usages etc.

It seems this new Pan feature is doing a similar thing
that the yencee project was doing in its earlier versions,
in that it is "counting encoded lines per article",
which is a wrong way to treat the 'slices' or 'chunks' of a file
in order to fit under the NNTP server's article size limit.

The basic thing here is to create those slices as a known _file_ size,
which is to say,
we need the original binary (decoded) data to be split precisely into that size.
We need the binary file-splits to match the PAR2 blocksize,
else their possible repair might be extremely difficult to calculate.

"Counting encoded lines" won't do this properly.

The actual binary data needs to be split _before_ it is encoded
(before uuencoded, before yEncoded, before MIME'd, etc.)
or in the view of the end-user/downloader, _after_ decoding.

That way, the posted encoded data,
when DECODED by the end-user (downloader),
will be in the expected size,
especially in the case if it needs to be repaired,
whereupon the number of PAR2 blocks can be easily determined.

In the history of the yencee project,
<https://sourceforge.net/projects/yencee/>,
I was able to convince the author to change this logic
(which was rather easy with the way his Perl script was written).
<https://sourceforge.net/tracker/?func=detail&aid=1427154&group_id=46988&atid=448128>
<https://sourceforge.net/project/shownotes.php?release_id=437775>

So please, before Pan is hacked-up even further,
please change this logic in the file-upload feature,
or else Pan uploaders might be blamed for "unusual" postings
if not worse.  ;)

In meantime, I've grown to trust yencee with my chosen parms & scripts etc.
over & above any other NNTP file-uploader mechanism.  ;)
I hope I can begin to trust this new feature in Pan, sooner or later, too.

As an aside, I'm hoping the yencee author will note my latest patch
that adds several (optional) headers to an upload-posting.
<https://sourceforge.net/tracker/?func=detail&aid=3111172&group_id=46988&atid=448129>
It's a work-in-progress, still seeking ways to make it more generic.
(I know he is very busy, he'll get to it as time permits & possible.)
I hope Pan will follow its own posting logic thoroughly
by attaching the "extra headers" as found
in the current "posting profile" & other places etc.
within the new file-upload feature, too.

I hope to have a few more writeups on the current new Pan works,
soon as I can get 'em typed-up.

Thanks for reading.