|
Ben,
Many thanks - just to note that this involved a *system* rather than
portable format save:
> 2) Delete all but 8 fields, reduces .sav file to 4 Mb
> 3) Save
process took approx. 1hr 40 mins (8:40-10:20 pm)
Regards,
Dave.
----- Original Message -----
Sent: Tuesday, August 05, 2008 6:12
PM
Subject: Re: PSPP Windows/SET TEMP/File
save times
Loading and, especially, saving to the portable file format
is
not well optimized. It is difficult to do in a fast and
portable
manner, because numbers in portable files are expressed in
an
awkward manner that is difficult to work with quickly.
It's
quite possible that this is the problem, especially given that
the
system file format worked much faster.
"David Purdy" <address@hidden>
writes:
> Correction: a file save with SET TEMP defined in WinXP has
produced no
> material improvement in lapsed time over a comparable file
save without SET
> TEMP defined (>90 minutes to save 9 Mb; see
earliest message):
>
> 1) Source SPSS .sav file 417 fields
x 510,121 records = 265 Mb
>
> 2) Delete all but 8 fields,
reduces .sav file to 4 Mb
>
> 3) Save process took approx.
1hr 40 mins (8:40-10:20 pm)
>
> PC = WinXP Pro SP3, 78 Gb
hard disk space available, 2048 Mb RAM (Dell
> Dimension 8300 3.2GHz
Pentium 4)
>
> Regards,
>
>
Dave.
>
> ----- Original Message -----
> From:
unknown-1
> To: David Purdy
> Cc: address@hidden
> Sent: Sunday,
August 03, 2008 9:37 PM
> Subject: Re: 0.6.0-10 PSPP windows - Queries
(WinXP)
>
> [cut]
>
>
2) Quicker 'Save' - I've had some mixed results, maybe influenced
by
> working with a large file (>500,000
records). After trying something
>
similar to the very slow save reported earlier (after deleting all
but
> about 7 fields), I got the impression that
the file was being created much
> more
quickly.
>
> Nice that is what I expected from using the temp
variable/directory.
>
>
[cut]
>
> ----- Original Message
-----
> From: David
Purdy
> To: address@hidden
>
Cc: Ben Pfaff [PSPP]
> Sent: Friday, August 01,
2008 11:08 AM
> Subject: Re: 0.6.0-10 PSPP
windows - Queries (WinXP)
>
> Ben,
>
> Many thanks for the
feedback.
>
> > 1) Very long file save time
?
>
> I was
using the GUI version - the disk didn't seem to be
especially
> active, nor PSPP using a lot of CPU
time (I think). I should mention
>
that the file was saved in the 'portable' format (9Mb/90mins) - I've
since
> saved the 'portable' file in the
'system' format instead (3Mb/2 mins), and
> the
'system' version also seems to load much more
quickly.
>
> [cut]
>
> ----- Original
Message -----
> From:
Ben Pfaff
> To: David
Purdy
> Cc: address@hidden
>
Sent: Friday, August 01, 2008 4:52
AM
> Subject: Re:
0.6.0-10 PSPP windows - Queries
(WinXP)
>
> "David Purdy"
writes:
>
> > 1) Very long
file save time ? - I'm using a large third party
SPSS
> .sav
file
> > (417 fields
x 510,121 records = 265 Mb), but am only interested in
7
> fields.
> > To ease the
processing load, I deleted the unwanted fields in
PSPP,
> but
the
> > resultant
.sav file took over 90 minutes to save (9 Mb). Are
there
>
any
> > settings that
could be adjusted to reduce the file save time
?
>
> Wow! That's a
long time. I don't know, off-hand, why it
would
> take that
long. Is the disk active during that time? Is
PSPP
> using a lot of
CPU time? Is this in the GUI or the
command-line
> syntax
version?
>
> Increasing the
workspace to a value larger than the size of
your
> file might help,
e.g. SET WORKSPACE=300000.
>
> [cut]
>
>
_______________________________________________
> Pspp-users mailing
list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/pspp-users
--
"I knew it was a bad sign when I drove through Hell (MI) on
Wednesday
and there was snow on the ground and all the standing water had
a
layer of ice on the surface... Yes folks, Hell has frozen
over
and my cable modem *still* wasn't working..." --Ed
Glowacki
|