In-place matrix operations in dynamically-linked c++ code

[Douglas Eck]

Paul Kienzle was nice enough to respond to my question about
memory usage in matrix operations. With his help I worked
up a sample bit of code to show the usefulness of doing
inplace matrix ops from time time.

Short story: if you're doing lots of matrix multiplications
over small matrices (e.g. a neural net) the limiting speed
factor can become the allocation and deallocation of
memory in matrix operations. If you want to, (and you are careful)
you can call the BLAS matrix operations directly and skip the
memory stuff done in the Matrix class's implementation
of, e.g., multiply.

If you don't want to run this code, here are some results.
On a Pentium Coppermine 850 with 256MB Ram
1000000 multiplies of a 50x15 matrix with a 15x1 matrix:
   Inplace took 4.601 user / .02 system seconds
   Regular took 8.500 user / .01 system seconds

I believe inplace operations might be a useful addition
to liboctave. I intend to write my own functions
as a step towards this (just syntactic sugar to make calling the BLAS
routines easier). I can post them later if they work.

As an aside, the reason I looked into this was that my
neural network memory footprint continues to grow
as the simulation goes on. I'm hoping that implementing
inplace matrix ops (inner and outer multiply, as well
as application of map) will drastically decrease this
growth.

We shall see.... in any case, with >20% of my time spent in
~Array2() it certainly can't hurt to treat the innermost
loop carefully in terms of (de)allocation.

Cheers,
Doug

--
Dr. Douglas Eck, http://www.idsia.ch/~doug
Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA)
Neural Networks, Rhythm Perception and Production, Dynamical Systems

tstInplace.tgz
Description: GNU Unix tar archive