[Octave-bug-tracker] [bug #39387] (Patches attached.) With large cell-arrays of numeric scalars, cell2mat is slow and memory-hungry.

[Olaf Till]

URL:
  <http://savannah.gnu.org/bugs/?39387>

                 Summary: (Patches attached.) With large cell-arrays of
numeric scalars, cell2mat is slow and memory-hungry.
                 Project: GNU Octave
            Submitted by: i7tiol
            Submitted on: Mon 01 Jul 2013 05:15:42 PM GMT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Performance
                  Status: None
             Assigned to: None
         Originator Name: Olaf Till
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: GNU/Linux

    _______________________________________________________

Details:

The patches will be attached as soon as I have the bug number for the commit
messages.
 
Slowness of cell2mat:


octave:5> c = zeros (10000000, 3);
octave:6> tic; c = num2cell (c); toc
Elapsed time is 1.89468 seconds.
octave:7> tic; d = cell2mat (c); toc
Elapsed time is 355.079 seconds.


The latter command produced signes of thrashing due to memory exhaustion
(severe disk access, slowing down of other applications).

(Same for reversed dimensions:


octave:5> c = zeros (3, 10000000);
octave:6> c = num2cell (c);
octave:7> tic; d = cell2mat (c); toc
Elapsed time is 438 seconds.

)

Slowness is partially due to a call, during argument checking:


octave:66> tic; cellfun ("isnumeric", c); toc
Elapsed time is 30.1 seconds.


which is made although there is no specialized method "isnumeric" in cellfun.

The rest of slowness and the thrashing seem to be due to the
cellfun("cat", ...)  calls, which are necessary for cell-arrays
containing non-scalar elements. The code strives to minimize the number of
cat() calls (within cellfun() calls) but seemingly has the sort order of the
dimensions mistaken, it should be descending (which was the intention also
according to the comments in the code). After correcting this, there is no
thrashing (or less?) and the function is faster:


octave:2> c = zeros (3, 10000000);
octave:3> c = num2cell (c);
octave:5> tic; d = cell2mat (c); toc
Elapsed time is 61.2661 seconds.


and with reversed dimensions


octave:6> c = zeros (10000000, 3);
octave:7> c = num2cell (c);
octave:8> tic; d = cell2mat (c); toc
Elapsed time is 55.0627 seconds.


The attached cell2mat changeset makes the above correction and
furthermore introduces a special case for scalar-only elements, where one
single cat() call can be used. This special case shortens duration from (here
already with the cellfun changeset (see below), so both are faster than
before):


octave:5> tic; cell2mat (c); toc
Elapsed time is 27.9482 seconds.


to


octave:6> tic; cell2mat (c); toc
Elapsed time is 9.9075 seconds.


I'd suggest applying this changeset to the stable branch.

The attached cellfun changeset adds the "isnumeric" special case to cellfun.
With this:


octave:7> tic; cellfun ("isnumeric", c); toc
Elapsed time is 0.326492 seconds.


(was 30 s before, see above), which adds to the speed enhancement of
cell2mat.

I'd suggest applying this changeset to the default branch.

Olaf





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39387>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/