[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

File processing advice

From: AlexG1
Subject: File processing advice
Date: Mon, 2 Mar 2009 06:29:54 -0800 (PST)


I need to write a script that executes the following operation:

1. Receive a filepath, 2 column indexes and 4 bound values (let's call them
2. For every line in the file, if (minX<line(index1)<maxX &&
minY<line(index2)<maxY) comment out the line, 
otherwise - just copy it.
3. Write the results to an output file.

I wrote the script but I'm getting pretty terrible performance, e.g. a file
of ~20000 lines takes about a minute to process (for comparison, the same
script with the same input takes less than 10 seconds in Matlab).

I'm pretty sure I didn't write it in the best way possible (though I still
can't understand why the huge difference from Matlab), so I'd like some
optimization advice. What I do now is the following (using fscanf, 
fprintf for reading and writing from\to file):

1. While didn't reach end of file:
        1.1 Read a line from the input file
        1.2 Check whether the values in the current line are inside the bounds:
                1.2.1 Yes - Write a comment character ('%') to the output file 
                1.2.2 No - do nothing
        1.3 Write the line itself and a newline character to the output file

I think copying the whole file to the output file path and then prepending
comments to each qualifying line would speed it up, but I don't know how to
do that without reading and writing the whole line (which would cancel out
any positive effect).

Any advice on how the speed can be improved will be highly appreciated.

Thank you,

View this message in context:
Sent from the Octave - General mailing list archive at

reply via email to

[Prev in Thread] Current Thread [Next in Thread]