Re: experimental support for SPV files now available

[Ben Pfaff]

On Sun, Dec 02, 2018 at 01:04:51PM -0600, Alan Mead wrote:
> On 12/2/2018 11:55 AM, Ben Pfaff wrote:
> >
> > Thanks for the report.  I ran into a previous issue where SPSS was
> > saving every space as a non-breaking space.  I guess it must use other
> > space variants sometimes too.  Weird.
> >
> > This seems to be specific to the particular SPV file, because after
> > converting a couple of SPV files to text I don't see that behavior.
> > Oddly, I don't seem to have the particular SPV file in question.  If you
> > send it to me, I'll fix the problem.
> 
> I can confirm that a recent (version 25) SPV file looks fine.

OK.  If you send this broken file, then I'll fix it.

> It sounds like from comments you made offline that the more examples
> that do not work (or have issues), the better for nailing down the
> conversion (since it has a lot of irregularities)?

Yes, absolutely.  My approach is corpus-based, and whenever I make a
change or bug fix I go back and check the corpus to make sure that I
didn't have obvious regressions.  Try it for everything you can
(respecting confidentiality) and it will get better as I incorporate new
files and fix the bugs.

> I also had no idea the service was so widely used... dozens of files per
> day (at least recently)? That's fantastic actually!

Since April 2018, which I guess is the last time I cleaned out the
input, I have collected about 14,000 submissions that total about 11 GB.
Most of these are .sav files, but .spv (about 3,000) and .spo files and
Excel spreadsheets and random .zip files and other things show up too.
Frankly it's an astonishing volume of data, over 1 GB/month.  It's been
great for making the PSPP .sav file reader reliable.