[Octave-bug-tracker] [bug #51871] load ascii-based numeric file is significantly slow

[Dan Sebald]

Follow-up Comment #3, bug #51871 (project octave):

push_back of std::vector:  OK, that's good.  I would guess this is the case
seeing the reduced time you cited.

I'm noting the extra step of:


Function read_mat_ascii_data()
{

  Function get_lines_and_columns()
  {
     Read the file line by line (get_mat_data_input_line()), exam number of
columns, see if it consistent. The validation of numerics is disabled
(check_numeric = false);
     Return number of columns and rows.
  }

  ASSIGN THE MEMORY FOR WHOLE NRxNC MATRIX.

  Read the file line by line (get_mat_data_input_line()), parse the numbers
one by one in a line, and put it in the matrix.
  AND FILL IN USING tmp.elem (i, j) = d; WHERE tmp IS CLASS Matrix.
}

Function get_mat_data_input_line()
{
  Return a data line, skips comments and white or empty lines.
}


OK, from your guess, where is the most time used?  Is it determining number of
rows and columns?  Is it getting the line, i.e., get_mat_data_input_line()?
because it has to look for comments and empty lines?  Is it the next loop back
from that?  In other words, what can be sped up here?

If it is determining number of rows and columns, then perhaps that could be
skipped and use v.push_back().  If it is getting the line, then I'm not so
sure.  If it is the loop that calls get_mat_data_input_line, then I'm not so
sure either.  However, another strategy to keep in mind is that
get_lines_and_columns() could also check if there are any comments, empty
lines.  If not, then your fast reading approach could be used; otherwise fall
back on the existing code.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?51871>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/