[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: reading jpegs faster
From: |
David Grundberg |
Subject: |
Re: reading jpegs faster |
Date: |
Fri, 22 Jan 2010 21:06:34 +0100 |
User-agent: |
Thunderbird 2.0.0.23 (X11/20090812) |
David Grundberg wrote:
John W. Eaton wrote:
On 22-Jan-2010, David Grundberg wrote:
| I'm reading 10 megapixel jpeg files alot. Unfortunately it me takes
9 | seconds to load such files with imread on the tip. I think that
is too | long time. Is someone experiencing the same problem as me?
Or does it | have to do with my configuration?
| | I've written a function file in C++ that loads the same file in
about 1 | second.
Doesn't the GraphicsMagick library ultimately use libjpeg? What
causes it to be slow? Rather than writing a special case for jpeg
files, I think it would be better to find out why imread is slow and
fix that. It's possible that would improve the performance of
Octave's imread function for all image formats, not just jpeg files.
jwe
I would guess it the slowness is due to extra copying/conversion being
done.
For example,
for (int y = 0; y < rows; y++)
{
idx(0) = y;
for (int x = 0; x < columns; x++)
{
idx(1) = x;
idx(2) = 0;
im(idx) = scale_quantum_to_depth (pix[i].red, depth);
idx(2) = 1;
im(idx) = scale_quantum_to_depth (pix[i].green, depth);
idx(2) = 2;
im(idx) = scale_quantum_to_depth (pix[i].blue, depth);
i++;
}
}
is slow and stands for the most of the time spent. I think it's a good
idea to use a general-purpose library like GraphicsMagick++, but as it
currently stands it's wasting a lot of cycles just converting between
char, int and double, and modifying index vectors. And the end result
is that we've converted uint8 to uint8. I did some ugly hacks to the
above loop but I wasn't particularly successful; got down to
disappointing 6 seconds.
There's also the problem with the Octave image convention, a backwards
way of storing images, where pixels are scattered throughout memory.
Maybe the postprocessing to Octave image conventions could be done
with more general purpose tools like Array::permute? Or is the current
approach with rearranging directly better?
GraphicsMagick++ way of representing pixels (PixelPacket *), a
blessing or a curse? It is a catch-all representation that will make
conversions obligatory where there would be an opportunity to use data
directly.
David
Okay. Now I got it down to 1.3 seconds. That's 130% compared to the
jpgread performance, instead of the previous 900%. Needs polish, but it
works.
Does anyone know of a good collection of image files of different types
(color depth, paletted, etc)? Only testing my camera's JPG files will
not cover a lot of different cases.
David
diff -r d307cba69817 src/DLD-FUNCTIONS/__magick_read__.cc
--- a/src/DLD-FUNCTIONS/__magick_read__.cc Tue Jan 19 16:55:47 2010 +0100
+++ b/src/DLD-FUNCTIONS/__magick_read__.cc Fri Jan 22 20:56:52 2010 +0100
@@ -35,7 +35,29 @@
#include <Magick++.h>
-unsigned int
+template <class P, unsigned int depth, unsigned int quantumdepth>
+inline P
+scale_quantum_to_depth (const Magick::Quantum& quantum)
+{
+ return (static_cast<P> (static_cast<double> (quantum)
+ / MaxRGB * ((1 << depth) - 1)));
+}
+
+template <>
+inline octave_uint8
+scale_quantum_to_depth<octave_uint8, 8, 8> (const Magick::Quantum& quantum)
+{
+ return static_cast<octave_uint8> (quantum);
+}
+
+template <>
+inline octave_uint16
+scale_quantum_to_depth<octave_uint16, 16, 16> (const Magick::Quantum& quantum)
+{
+ return static_cast<octave_uint16> (quantum);
+}
+
+inline unsigned int
scale_quantum_to_depth (const Magick::Quantum& quantum, unsigned int depth)
{
return (static_cast<unsigned int> (static_cast<double> (quantum)
@@ -190,7 +212,7 @@
return output;
}
-template <class T>
+template <class T, class P, unsigned int D>
octave_value_list
read_images (const std::vector<Magick::Image>& imvec,
const Array<int>& frameidx, unsigned int depth)
@@ -269,31 +291,41 @@
case Magick::PaletteType:
case Magick::TrueColorType:
- idim(2) = 3;
- im = T (idim);
- for (int frame = 0; frame < nframes; frame++)
- {
- const Magick::PixelPacket *pix
- = imvec[frameidx(frame)].getConstPixels (0, 0, columns, rows);
+ {
+ idim(2) = 3;
+ im = T (idim);
+ P *vec = reinterpret_cast<P *> (im.fortran_vec ());
- int i = 0;
- idx(3) = frame;
+ for (int frame = 0; frame < nframes; frame++)
+ {
+ const Magick::PixelPacket *pix
+ = imvec[frameidx(frame)].getConstPixels (0, 0, columns, rows);
- for (int y = 0; y < rows; y++)
- {
- idx(0) = y;
- for (int x = 0; x < columns; x++)
- {
- idx(1) = x;
- idx(2) = 0;
- im(idx) = scale_quantum_to_depth (pix[i].red, depth);
- idx(2) = 1;
- im(idx) = scale_quantum_to_depth (pix[i].green, depth);
- idx(2) = 2;
- im(idx) = scale_quantum_to_depth (pix[i].blue, depth);
- i++;
- }
- }
+ int i = 0;
+ P *rbuf, *gbuf, *bbuf;
+ rbuf = vec;
+ gbuf = vec + rows * columns;
+ bbuf = vec + rows * columns * 2;
+ for (int y = 0; y < rows; y++)
+ {
+ for (int x = 0; x < columns; x++)
+ {
+ *rbuf = scale_quantum_to_depth<P, D, QuantumDepth>
(pix[i].red);
+ *gbuf = scale_quantum_to_depth<P, D, QuantumDepth>
(pix[i].green);
+ *bbuf = scale_quantum_to_depth<P, D, QuantumDepth>
(pix[i].blue);
+ i++;
+ rbuf += rows;
+ gbuf += rows;
+ bbuf += rows;
+ }
+ rbuf -= rows * columns - 1;
+ gbuf -= rows * columns - 1;
+ bbuf -= rows * columns - 1;
+ }
+
+ // Next frame.
+ vec += rows * columns * 3;
+ }
}
break;
@@ -426,17 +458,23 @@
switch (depth)
{
case 1:
- output = read_images<boolNDArray> (imvec, frameidx, depth);
+ output = read_images<boolNDArray, bool, 1> (imvec, frameidx, depth);
break;
case 2:
+ output = read_images<uint8NDArray, octave_uint8, 2> (imvec,
frameidx, depth) ;
+ break;
+
case 4:
+ output = read_images<uint8NDArray, octave_uint8, 4> (imvec,
frameidx, depth) ;
+ break;
+
case 8:
- output = read_images<uint8NDArray> (imvec, frameidx, depth) ;
+ output = read_images<uint8NDArray, octave_uint8, 8> (imvec,
frameidx, depth) ;
break;
case 16:
- output = read_images<uint16NDArray> (imvec, frameidx, depth);
+ output = read_images<uint16NDArray, octave_uint16, 16> (imvec,
frameidx, depth);
break;
case 32: