[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #49630] importdata very slow when read large f
From: |
Philip Nienhuis |
Subject: |
[Octave-bug-tracker] [bug #49630] importdata very slow when read large files of text containing data |
Date: |
Thu, 17 Nov 2016 12:25:26 +0000 (UTC) |
User-agent: |
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0 |
Update of bug #49630 (project octave):
Item Group: Matlab Compatibility => Performance
Status: None => Confirmed
Release: 4.0.0 => dev
Summary: importdata cannot read large files of text
containing data, whereas matlab does => importdata very slow when read large
files of text containing data
_______________________________________________________
Follow-up Comment #1:
The command works fine in the development version. But I agree that
importdata.m should work a bit more efficient.
A little debug session shows that in L. 168 and up it doesn't read just "a
first few rows" but rather reads all rows. I waited 5 minutes (no response,
Octave busy reading) and then had a lunch break. After that Octave had
finished reading 217952 labels.
Then, in L.208 & up it again reads & parses the entire file.
I haven't tried beyond L.235 and up, as I noted that the input file is to be
read another time (w. dlmread) & parsed again.
FYI, it is much more efficient to read these files using textscan
>> fid = fopen ("imageLabels.txt")
fid = 3
>> tic; a = textscan (fid, "%s"); toc
Elapsed time is 4.40244 seconds.
>> fclose (fid)
ans = 0
>> more on
>> a{1}
ans =
{
[1,1] = roofTops
[2,1] = forest
[3,1] = forest
:
(Title, Item group & release adapted)
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?49630>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/