|Subject:||"ERROR I/O" on a large database on version 0.7.9 Win7 binary|
|Date:||Fri, 1 Jun 2012 19:53:12 +0000|
I would like to run a 2GB CSV database (3.4 million cases, 159 variables) on Win7.
Somewhere between 38.2MB (51450 lines) and 38.2 MB (51475 lines), PSPP breaks down -- it can't import the dataset. I'm guessing that the number is based on something wrong with the windows binary or some temp access problem. Instead of the dataset being truncated like I've seen in other posts, though, every cell in the GUI is simply blank (or, in the command line, "ERROR I/O"). Running as administrator doesn't change this, and neither does obtaining user permission to access \Windows\Temp (which makes sense, since running as admin didn't work). Here's some system information:
OS: Windows 7 Home Premium 64-bit
HDD: ~200 free GB
2 cores @ 3.4 GHz, 12GB RAM
PSPP: psppire.exe 0.7.9 March 15, 64 bit from http://pspp.awardspace.com/, though I've tried probably 10 different versions and gotten the same result, including the most recent version and the other builder's version.
When I run command line I get no error until LIST is run; then I get "Error I/O" and also an error writing to tmp (could not see exact syntax using LIST since the table began outputing thousands of lines of "ERROR I/O"). I get a large number of these in the syntax:
C:\Users\User\file\path\to\csv\database.csv:87.183-87.187:warning: Data for variable VAR036 is not valid as format F: Field contents are not numeric.
and one of these:
note: Warnings (101) exceed limit (100). Syntax processing will be halted.
The actual data is under a Data Use Agreement so I can't post it here, but here's the SPSS load program:
I've tried making a non-sensitive information version of the data, and seem to have succeeded, in that I get the same results. The first ten variables are similar to DX1-10, and the rest of the variables are just there to bulk up the file size. Here it is:
|[Prev in Thread]||Current Thread||[Next in Thread]|