[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Slow Processing Issue

From: Fritz Sonnichsen
Subject: Re: Slow Processing Issue
Date: Wed, 12 Jul 2017 12:49:00 -0400
User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0

Thanks all!

I just looked at Francesco's post (busy week here) and wrote up the script using textread. I had avoided it because matlab is obsoleting textread but I am more than happy to use it. This solved the problem--it is much faster. I ran 135,000 records in 13.6 seconds.

The pertinent code area was:
[C,R,V]= textread (filename,"%*s,%f,%*f,%*f,%*f,%f,%f,%*f,%*f,%*s,%*s",'delimiter',',');

I had to use the "delimiter as I am using CSV. The "asterik" format was handy to avoid using up memory.

Thanks for the help

On 7/12/2017 8:16 AM, siko1056 wrote:
Fritz Sonnichsen wrote
I have a very simple script shown below. I am processing a small
file-about 50,000 lines of 60 bytes each. The file loads just fine but
the "for" loop takes around 15 minutes on a laptop that generally does
not have performance issues. I recall that early versions of matlab had
some type of "file reallocation" recommendation to speed things up.
    Should Octave be able to do this script in a reasonable amount of time?
    Am I doing anything wrong here?


clear all
    workfile  = fileread
    workfile = strsplit (workfile, "\n");
    disp("WORKFILE LOADED");fflush(stdout);
    for i = 1:length(workfile)
      Clog(i,:) = strtrim (strsplit (workfile{i}, ","));
      %if mod(i,1000)==0 display(i); fflush(stdout);end;
Hello Fritz,

Did you give the approach from Francesco Potortì [1], using textread, a try?
Otherwise you could try to pre-allocate the cell-array Clog of my adapted

more off # instead of fflush, why are you using fflush anyway??
clear all
workfile = fileread ('C:\Users\fsonnichsen\Desktop\conduct\minicom.cap');
workfile = strsplit (workfile, "\n");

# pre-allocation
N = length(workfile);
Clog = cell (N,4);

for i = 1:N
  Clog(i,:) = strtrim (strsplit (workfile{i}, ","));
  %if mod(i,1000)==0 display(i); end;


View this message in context:
Sent from the Octave - General mailing list archive at

Help-octave mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]