very long string support

[Ben Pfaff]

I took a look at your changes for supporting very long strings.
It looks good, except that I don't understand the choice of where
to "mangle" and "demangle" strings.

Here's what I understand about the system file format for very
long strings, based on what you checked in.  The basic system
file format only supports strings up to 255 bytes long.
Therefore, for compatibility very long strings are broken up into
255-byte segments with unique names, and a system file extension
record explains how to paste those segments back together.

I think that what you checked in changed the *internal* PSPP
storage of strings so that it stores each 255 bytes of a string
in 256 bytes, putting a space in the 256th byte.  In other words,
it changes PSPP internals to match the stupid but compatible
format of SPSS system files.

To me that looks like a mistake.  System files have to be
compatible, so they have a stupid format for very long strings.
But that's no reason to use that stupid format internally and
then have to deal with it potentially all over (it doesn't look
to me like you fixed up everything that can use very long
strings, e.g. AGGREGATE, and I'd rather that we not have to).
Instead, we should translate between the obvious, normal format
and the stupid one on input and output, and then all the internal
code can stay the way it is.

Most system files won't need translation at all, because most of
them don't have very long strings, so on input this can just be
another thing that happens in the "slow path" in sfm_read_case().
On output we don't currently have a "slow path" (except for
compressed data) so we'll have to add something.

Does all of that make sense?
-- 
Ben Pfaff 
email: address@hidden
web: http://benpfaff.org