[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Reading data from large ascii-file

From: Peter Norlindh
Subject: Re: Reading data from large ascii-file
Date: Tue, 19 Apr 2011 12:05:03 +0200

Thank you Marco!  I have now tried to read the data in bigger chunks, and that seems to have sped things up a bit:)
Currently, I'm waiting for dlmread(FILE) to read the entire file.  It's been running for about one hour now and I expect it to continue for quite some time.  Are there ways to decrease the reading time even further?

On Tue, Apr 19, 2011 at 10:09 AM, marco atzeri <address@hidden> wrote:
On Tue, Apr 19, 2011 at 9:41 AM, Peter Norlindh  wrote:
> Hi,
> I am analyzing data from a 400MB ascii-file.  There are about 30 million
> data points in the file and the execution takes forever.
> Currently the program reads and processes the file element-by-element ( val
> = dlmread(fileName, "emptyvalue", [i, col, i, col]) ) and I suspect that
> this is a very unafficient way to do it.  Could you suggest a better
> approach?

surely not reading one element at time as you are doing with [i, col, i, col]

The RANGE parameter may be a 4-element vector containing the upper
    left and lower right corner `[R0,C0,R1,C1]' where the lowest index
    value is zero.

if the data is already in matrix form, you should read all the matrix
in one pass
with just

DATA = "" (FILE)

> Regards,
> hpon


reply via email to

[Prev in Thread] Current Thread [Next in Thread]