[Octave-bug-tracker] [bug #30468] dlmread - performance problems

[Rik]

Update of bug #30468 (project octave):

                Category:                    None => Interpreter            
              Item Group:                    None => Other                  
                  Status:                    None => Confirmed              

    _______________________________________________________

Follow-up Comment #1:

Yes, dlmread is absolutely atrocious.  The file is written in C++, so one
would expect it to be of relatively high performance, but it isn't.

I'm attaching a small Perl script which makes a test file which is a matrix
of 100,000 rows X 5 columns.  This was useful as input for testing.

Incidentally, Octave's normal 'load' command can handle tab-separated data. 
Until dlmread is fixed, you will have better luck with that.  It won't handle
header rows, however, so you would need to delete those.

For example, I benchmarked Octave's 'load' command on the 100K test matrix.

tic; x = load("matrix.csv"); bm = toc
bm =  0.74381

Less than 1 second to load the data.  After that you could simply throw out
the column you don't need.

x(:,1) = [];


(file #21005)
    _______________________________________________________

Additional Item Attachment:

File name: mk_file.pl                     Size:0 KB


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?30468>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/