[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Import large field-delimited file with strings and numbers

From: Philip Nienhuis
Subject: Re: Import large field-delimited file with strings and numbers
Date: Wed, 10 Sep 2014 14:46:22 -0700 (PDT)

Philip Nienhuis wrote
> Joao Rodrigues wrote
>>>> I need to import a large CSV file with multiple columns with mixed
>>>> string and number entries, such as:
>> <snip>
> Why do you need to break it up using csv2cell? AFAICS that reads the
> entire file and directly translates the data into "values" in the output
> cell array, using very little temporary storage (the latter quite unlike
> textscan/strread).
> It does read the entire file twice, once to assess the required dimensions
> for the cell array, the second (more intensive) pass for actually reading
> the data.
> BTW, on my TODO list is an old (2+ years?) entry: adding a "headerlines"
> parameter to csv2cell..... (but OK my TODO list is looong)

FYI, I've just added this headerlines parameter to csv2cell; I needed it
myself as well, so your post was the final nudge to get me started with it.
I'll push it later this week.

Along the way I've also promoted the row counter type from int to long (int)
- supposedly that'll make a difference with some extremely big data files

This will be part of a (soon to come) io-2.2.4 release.


View this message in context:
Sent from the Octave - General mailing list archive at

reply via email to

[Prev in Thread] Current Thread [Next in Thread]