Question on performance, coding style and competitive software

[Alois Schlögl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


As some of you might know, my other pet project besides Octave, is
BioSig http://biosig.sf.net. BioSig is designed in such a way that it
can be used with both, Matlab and Octave. Mostly for performance reason,
we cannot abandon support for Matlab [1,2]. Octave is a viable
alternative in case the computational performance is not important. In
order to decide on the future strategy of BioSig, I hope to get answers
on the following questions:

1) Core development of Octave:
At the meeting of Octave developer in 2006, the issue was raised
that the Octave is about 4 to 5 times slower than Matlab [1]. (I
repeated the tests just recently, the results are attached below, and
show a difference of factors up to 13, average ~5) This issue is most
relevant for large computational jobs, were it makes a difference
whether a specific task takes 1 day or 5 days. Is anyone working to
address this problem? Is there any hope that the performance penalty
becomes smaller or will go away within a reasonable amount of time ?

2) Coding style:
Octave understands a superset of commands compared to matlab, and it
seems the current policy is to enforce the "octave style" and make the
use of toolboxes incompatible for a use with Matlab. Is not it sensible
to write platform-neutral applications ? Specifically, is not it in our
own interest (wider usage make the code more robust) that matlab users
are not "forced" to buy additional toolboxes but can use open source
toolboxes e.g. from octave-forge?

3) Scope of Octave and Octave-Forge:
Open source software has its own merit, but sometimes also other factors
(e.g. additional costs in hardware, energy supply and cooling systems,
energy efficiency = "green computing") need to be considered. Given the
fact that octave-core is currently slower for some tasks, it is worth
considering to use proprietary mat-engine. The question is whether
Octave and Octave-forge should provide support of toolboxes for matlab
users too, or whether these users should go somewhere else? What do you
think ?


    Alois


[1] http://arxiv.org/abs/cs/0603001v1
[2] benchmark from biosig (bench_biosig.m)
http://biosig.cvs.sourceforge.net/viewvc/biosig/biosig/demo/bench_biosig.m?revision=1.7&view=markup

In order to repeat the benchmark, you need biosig for octave and matlab,
the tsa-toolbox and the nan-toolbox from octave-forge. The results are
obtained from an AMD64 quad core machine with 64bit Ubuntu 8.10.


Version: 3.1.54+
cputime  toc[s] task
================================
  2.244   2.853 sload l1.gdf(#1)
 77.105  78.547 wackermann
  1.988   1.993 barlow
  2.128   2.133 hjorth
  0.992   0.994 TDP
200.065 200.119 aar #1
195.428 195.439 aar #2
197.860 197.862 aar #3
 65.684 519.944(#2) findclassifier barlow LD3
 66.336  66.333 findclassifier hjorth LD3
 66.240  66.251 findclassifier aar LD3
 65.748  65.753 findclassifier Wackermann LD3
 65.760  65.759 findclassifier TDP LD3
- -----------------------------------
1007.58 1463.98(#2) total


Version: 7.6.0.324 (R2008a)
cputime  toc[s] task
================================
  0.150   0.148 sload l1.gdf(#1)
  1.110   1.096 bandpower
 12.800  12.688 wackermann
  1.550   1.471 barlow
  1.600   1.477 hjorth
  1.290   1.053 TDP
 40.370  40.334 aar #1
 39.810  39.782 aar #2
 40.040  39.823 aar #3
  7.020   4.979 findclassifier bp LD3
  7.440   5.010 findclassifier barlow LD3
  7.550   5.093 findclassifier hjorth LD3
  7.930   5.205 findclassifier aar LD3
  7.860   5.230 findclassifier Wackermann LD3
  6.600   4.844 findclassifier TDP LD3
- ------------------------------
196.400 179.984 total

Remarks
(#1) The differences in "sload l1.gdf" are mostly to the fact that a
mex-version was used in Matlab, while Octave uses an m-script.
(#2) waiting for some key pressing.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAknt8FYACgkQzSlbmAlvEIi+YgCgopPkViJplWhYcTVS1gVRGTFc
kEoAnjfXUqOGcVZ7pz5f91gA4MSKSccX
=kcr2
-----END PGP SIGNATURE-----