[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: incremental read of gzipped matrix

From: Markus Mützel
Subject: Re: incremental read of gzipped matrix
Date: Sun, 8 Dec 2019 23:02:03 +0100

Am 08. Dezember 2019 um 22:30 Uhr schrieb "Andreas Weber":
> Am 08.12.19 um 19:25 schrieb "Markus Mützel":
> > Octave has the gzifstream (in zfstream.h) which might be useful for your 
> > purpose.
> I've used this before and only got an improvement by factor 2.
> The factor 2 is because load currently parses the inputfile twice: the
> first run is to detect number of columns and rows, then the Matrix is
> allocated then parsed the second time.

Do you know why gzifstream is so much slower compared to your code? Given that 
you measured a 10 fold performance increase, there might be room for a factor 
of 5 (taking the double parsing into account).

> In the meanwhile I've enabled incremental reads in load_gz which is very
> important for me if I periodically poll an increasing .gz file.
> -- Andy

reply via email to

[Prev in Thread] Current Thread [Next in Thread]