[Monotone-devel] Performance accounting and testing patch

[Eric Anderson]

Summary: The attached patch adds in memory usage, copys and malloc
accounting to monotone, and a repeatable performance test for
evaulating CPU usage, memory usage, copies, and mallocs. Sample output
is shown at the bottom after the detailed discussion.

The patch needs to be applied with a -p0 in order to preserve the
directory location of two new files.

The ChangeLog entry (included in the patch):

2005-08-09  Eric Anderson  <address@hidden>

        * main.cc: allow optional malloc, memcopy, resource 
        usage accounting, and a sub-process to track memory usage
        * tests/perf-test.sh: A repeatable performance test harness
        * tests/parse-accounting.pl: A script that parses the accounting
        output into a nice tabular format

Detailed discussion: 

The memory accounting is handled by forking a separate process that
monitors the current size and resident size of the parent process
every 5-10 ms.  This is necessary because Linux doesn't include
counters that determine the maximum size, and the number of minor page
faults is not an accurate estimator of the maximum size because the
malloc implementation will release memory when it is no longer needed.
This also means that just checking the resident size at termination is
not sufficient as all of the memory will have been returned and the
resident size will be very small.

The copy accounting is handled by writing a replacement memcpy, and
memmove functions that call bcopy to do the actual copy after
performing their accounting.  bcopy() is not used because it is the
deprecated copy interface, but it has the same semantics as memmove
which makes this replacement safe.

The malloc accounting is handled by writing a replacement malloc
function that calls __malloc to do the actual malloc after doing the
accounting work.  It has a flag that can be set to show the mallocs
over a particular size, which is useful for debugging where in the
program the large malloc is happening.  You could set a conditional
breakpoint on malloc, but this is implemented so slowly as to make it
useless given the large number of mallocs that occur during execution.

The perf-test.sh script creates a repeatable directory containing a
set of files and then uses the monotone binary under test to add the
files, commit the files, checkout the files, and serve/pull the files
from one database to another.  Technically the files are not
repeatable as a few random files are included, but the purpose of the
random files is simply to be non-compressible files of a particular
size.

The perf-test.sh script uses the parse-accounting.pl script to read in
the output from the monotone run and present it in a nice tabular
format.

perf-test.sh currently runs the following tests, assuming someone can
generate a repeatable set of files, it is easy to add additional tests
(one function to create the files to test on, and one line to add in
the new test).

   - zero_small: a 10KiB all zero file
   - zero_large: a 100MiB all zero file
   - random_medium: a 10MiB random file
   - halfzero_large: a 100MiB file, 50MiB of zeros and 50MiB of random data
   - random_large: a 100MiB random file
   - monotone: all the files in the 0.19 release
   - mt_multiple: all the files in the 0.10-0.22 releases
   - mt_bigfiles: each of the 0.10-0.22 releases concatenated to form a 
     single file for each release
   - everything: all of the above at the same time

Monotone-0.22 is not efficient enough to complete the "everything"
test in a reasonable time.  The next patch that I submit will fix this
problem.

59: ./tests/perf-test.sh  
Test CPU: Intel(R) Pentium(R) M processor 1700MHz
monotone 0.22 (base revision: 28058ae3e850229a5d8fae65415cbbf82b435377)
                                     Maximum (MiB)     Copied   Malloc
     *Test*     Operation  CPU(s)   Size  Resident     (MiB)    (MiB)
--------------- ---------  ------  -------  -------   -------  -------
zero_small      add files    0.02     7.29     2.75       0.3      1.0
zero_small      commit       0.28     7.62     4.07       8.5     10.8
zero_small      checkout     0.03     7.36     3.52       0.4      1.3
zero_small      serve        0.25     9.38     4.78       5.5     10.4
zero_small      pull         0.05     9.38     4.71       0.6      2.0

zero_large      add files    3.91   420.42   414.09    1447.6    101.0
zero_large      commit       8.79   308.04   204.42     711.7    317.0
zero_large      checkout     4.45   208.66   204.50     401.1    208.2
zero_large      serve        7.23   210.41   205.79     606.6    221.1
zero_large      pull         6.76   310.55   205.79     602.6    310.5

random_medium   add files    0.25    51.67    45.34     126.3     11.0
random_medium   commit       2.79    73.93    69.79     196.8    114.9
random_medium   checkout     1.94    56.67    39.18      77.5     64.6
random_medium   serve        3.50    58.37    46.64     162.9    107.8
random_medium   pull         2.53    85.85    80.96     229.2    150.0

halfzero_large  add files    3.77   414.81   409.91    1739.4    101.0
halfzero_large  commit      18.07   413.28   409.19    1302.4    660.2
halfzero_large  checkout    12.35   345.79   274.10     586.2    413.2
halfzero_large  serve       21.20   347.49   307.53    1091.7    583.2
halfzero_large  pull        17.14   465.63   460.70    1444.6    868.9

random_large    add files    3.75   414.81   409.33    1963.3    101.0
random_large    commit      28.71   651.73   647.56    1894.2   1038.4
random_large    checkout    20.68   480.88   341.64     771.3    616.2
random_large    serve       35.50   482.55   407.45    1577.2    940.2
random_large    pull        28.12   753.71   748.77    2286.6   1459.7

monotone        add files    0.38     9.23     4.45      26.5     11.1
monotone        commit       3.33    12.74     9.06     131.5    109.5
monotone        checkout     1.57    19.60    15.68      55.8     45.3
monotone        serve        3.04    20.68    15.76     122.4     99.4
monotone        pull         3.41    15.18    10.32     138.8    137.0

mt_multiple     add files    4.86    12.29     7.40     238.9    103.5
mt_multiple     commit      29.04    39.20    35.25     893.7    678.6
mt_multiple     checkout    14.84    25.71    21.79     477.1    367.8
mt_multiple     serve       10.12    19.99    14.95     320.0    261.0
mt_multiple     pull        88.47    53.00    37.57   30291.3  20461.8

mt_bigfiles     add files    3.71    73.76    69.06    1318.2    106.8
mt_bigfiles     commit      17.61    50.80    42.32    1141.2    466.4
mt_bigfiles     checkout    10.28    50.20    41.49     519.6    261.2
mt_bigfiles     serve       20.97    52.14    42.58     934.4    404.9
mt_bigfiles     pull        17.24    56.63    47.30    1167.3    608.4

load everything..........
everything      add files   19.94   500.49   495.10    6707.3    519.4
everything      commit     104.94   658.30   654.24    6098.8   2997.4
everything      checkout    68.16   497.13   392.59    2831.1   1693.8
[ aborted as it had >30minutes of CPU time and from the DB size was 
  < 50% done; a second run killed after 10 minutes of CPU time had 
  copied 282GiB, and had malloced 208GiB so far. ]

accounting.patch
Description: Binary data