Re: profiling Octave

[Rik]

On 08/09/2019 02:00 PM, Dmitri A. Sergatskov wrote:

On Fri, Aug 9, 2019 at 3:27 PM Rik <address@hidden> wrote:

On 08/09/2019 12:16 PM, Carnë Draug wrote:

On Fri, 9 Aug 2019 at 05:30, Rik <address@hidden> wrote:

I wanted to look to look empirically at the slow down that has occurred
over time with Octave.

[...]

Any ideas?

Have we actually established that Octave has indeed been running
slower over time?  There's been a lot of talk about it, but all the
measurements I have seen where by running the test suite and not a
real Octave program.

I think that before worrying about what tools to check where the
performance drop is, maybe we should get a set of programs that can be
used to confirm this slowdown.

For what is worth, I too have the feeling that Octave has been getting
slower over time, I just haven't actually got around to confirm this
with proper methods.

Always good to check assumptions, but unfortunately I have been testing regularly with the same code.  I didn't start out intending this to be a performance benchmark so it is rather limited, but I do have historical data captured with it.

The m-file is tst_for_loop.m and is the single line:

a = 1; b = 1; tic; for i=1:1000; for j=1:1000; a = a + b + 1.0; end; end; toc

Because I have changed computers over time I have a segmented profile history.  For the distant past I had

Version

3.2.4

3.4.3

3.6.4

3.8.0

3.8.1

Mean

2.24752

1.71794

2.96738

3.45597

3.16926

Right now I see

4.2.1

4.4.1

5.1.0

dev (6.1.0)

1.08590

1.38079

1.37580

1.41128

Part of the problem is that we have regression testing for functionality, but no regressions tests for performance.  Thus, we can accidentally change Octave's performance drastically without noticing.  This bug report captured one instance in which the performance decline was noted, and because it was noted it got fixed (https://savannah.gnu.org/bugs/?func=detailitem&item_id=52809).

--Rik

The code for Toeplitz matrix is:

cumulate = 0; b = 0;
for i = 1:runs
  b = zeros(620, 620);
  tic;
    for j = 1:620
      for k = 1:620
        b(k,j) = abs(j - k) + 1;
      end
    end
  timing = toc;
  cumulate = cumulate + timing;
end
timing = cumulate/runs;
times(4, 3) = timing;

So it is mostly  loops.

Dmitri.

--

I changed the code slightly to make this benchmark file, bm_toeplitz.m

runs = 5;

cumulate = 0; b = 0;
for i = 1:runs
  b = zeros(620, 620);
  tic;
    for j = 1:620
      for k = 1:620
        b(k,j) = abs(j - k) + 1;
      end
    end
  timing = toc;
  cumulate = cumulate + timing;
end

## Total time
cumulate

Results are the same as the simpler loop test, about a 25% slowdown from 4.2 to 4.4.

4.2.1	4.4.1	5.1.0
10.544	13.052	13.481

--Rik