Re: Loading ASCII data files quickly

[Christoph Ellenberger]

I usually use sed to straighten the data and then do the load thing....
something like:

cmd=['sed -n -e "/^ *[0-9]/p" ' fname  ">" Tfname];
if (system(cmd,1))
   error("Fehler 1");
end
    
xx=load(Tfname);


where fname is the filename and tfname is some temporary filename which I 
remove afterword with....

cmd=["rm " Tfname];
if (system(cmd,1))
    error("Fehler 4");
end

but this depends on what structure do you want afterward. I sometimes have the 
coulumnames in the file as well and create a cellarray with the datapoints 
called Data.X, Data.Y etc. as well.

Hope this helps
Christoph

-------- Original-Nachricht --------
> Datum: Mon, 10 Aug 2009 16:24:07 +0200
> Von: Matthias Brennwald <address@hidden>
> An: address@hidden
> Betreff: Loading ASCII data files quickly

> Dear all
> 
> I've got a few data files with a short header followed by two colums  
> of data (see example below). The header data is marked by '*' signs,  
> like this:
> 
> ------------------------------
> * TMD data written by MATAA on 10-Aug-2009 15:37:39
> * MSW, 0 deg., no filter
> 0     -6.49888e-08
> 1.04167e-05   -1.77097e-07
> 2.08333e-05   2.53872e-07
> 3.125e-05     9.02216e-07
> 4.16667e-05   1.13457e-06
> 5.20833e-05   6.72286e-07
> 6.25e-05      -1.5686e-07
> 7.29167e-05   -6.71451e-07
> 8.33333e-05   -5.46739e-07
> 9.375e-05     -7.39792e-08
> 0.000104167   1.52549e-07
> 0.000114583   -9.41797e-08
> .
> .
> .
> ------------------------------
> 
> I tried to read line by line using fgetl(...) into a string. If the  
> line starts with an '*', it is treated as header line. Otherwise it is  
> transformed into numbers using str2num(...). This line-by-line  
> procedure works, but is too slow for large files. I believe it would  
> be much more efficient to read only the header lines using this  
> method, and then use a file command that loads the rest of the data in  
> one step. Any hints or ideas how I can do this?
> 
> Matthias
> 
> 
> ----
> Matthias Brennwald, Käferholzstrasse 173, CH-8046 Zürich, +41 44 364  
> 17 03
> 
> 
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://www-old.cae.wisc.edu/mailman/listinfo/help-octave

-- 
 

GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01