[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Octave-patch-tracker] [patch #8140] Speed up importdata() ASCII CSV pro

From: Dan Sebald
Subject: [Octave-patch-tracker] [patch #8140] Speed up importdata() ASCII CSV processing using dlmread() as core
Date: Sun, 01 Sep 2013 23:31:00 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0 SeaMonkey/2.15

Follow-up Comment #9, patch #8140 (project octave):

I rolled back to a few days ago to compile octave and download the latest
importdata.m.  Looks nice.  I'll have to look through some of the regexp, etc.
mods to see the improvements at some other time.

The CR/LF case works, at least on Unix.  I suspect you are worried that
creating the test file:

fputs (fid, "3.1t-7.2t0rn0.012t6.5t128");

will create a file on Windows systems in which the linebreaks are CR-CR-LF and
the test will fail.  However, we might be able to change the way in which the
data file is generated by using explicit output data 0x0D (CR) and 0x0A (LF).

fprintf (fid, "3.1t-7.2t0%c%c0.012t6.5t128", 0x0D, 0x0A);

fprintf (fid, "3.1t-7.2t0%c0.012t6.5t128", 0x0A);

So we know we can generate files that are sure to have CR, or LF, or CR-LF. 
The question is whether these can all be handled properly on all platforms. 
If dlmread() can do so, then I see no problems with having these tests in
place.  If in the future dlmread() does not support these, then the tests can
be removed.  Both of the above pass the tests.  This one:

fprintf (fid, "3.1t-7.2t0%c0.012t6.5t128", 0x0D);

fails.  (We knew that.)

I looked at dlmread.  It is using "getline (*input, line);" to get a line of
the file.  I think getline is a C or C++ library function.  We could program a
fix for the last case easy enough by searching for the first CR or LF and
check whether the line delimiter is CR, LF or CRLF.  Then we can use a
delimiter as part of the getline() routine:


I think if we support CR, LF and CRLF we are preserving compatibility whatever
the case.


Reply to this item at:


  Message sent via/by Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]