Re: Import large field-delimited file with strings and numbers

[Philip Nienhuis]

Philip Nienhuis wrote
> 
> Joao Rodrigues wrote
>>>> I need to import a large CSV file with multiple columns with mixed
>>>> string and number entries, such as:
>> <snip>
> Why do you need to break it up using csv2cell? AFAICS that reads the
> entire file and directly translates the data into "values" in the output
> cell array, using very little temporary storage (the latter quite unlike
> textscan/strread).
> It does read the entire file twice, once to assess the required dimensions
> for the cell array, the second (more intensive) pass for actually reading
> the data.
> 
> BTW, on my TODO list is an old (2+ years?) entry: adding a "headerlines"
> parameter to csv2cell..... (but OK my TODO list is looong)

FYI, I've just added this headerlines parameter to csv2cell; I needed it
myself as well, so your post was the final nudge to get me started with it.
I'll push it later this week.

Along the way I've also promoted the row counter type from int to long (int)
- supposedly that'll make a difference with some extremely big data files
:-)

This will be part of a (soon to come) io-2.2.4 release.

Philip



--
View this message in context: 
http://octave.1599824.n4.nabble.com/Import-large-field-delimited-file-with-strings-and-numbers-tp4666380p4666456.html
Sent from the Octave - General mailing list archive at Nabble.com.