[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Need a faster way of reading files.

From: Ole Jacob Hagen
Subject: Need a faster way of reading files.
Date: Wed, 2 Jun 2010 08:23:54 -0700 (PDT)


I have large data files, with 90 columns. Typical size is 600-700 MB.
Data is stored in ascii format. This is data collected for 7200 s, with
sampling interval 100 Hz. 

I am using data = load('mydata.txt'), but this is very very slow, compared
to SciPy. 
SciPy uses one minute, while Octave is using approx 5-10 minutes. Haven't
measured time consumption, but can come back with that one. ;-) 

I've tried to use fread, but the data is not recognizable at all. 
I know how many columns I have in my data file. 

How can I accelerate my data loading into memory in Octave?

Best regards, 

Ole J. Hagen 

View this message in context:
Sent from the Octave - General mailing list archive at

reply via email to

[Prev in Thread] Current Thread [Next in Thread]