[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Monotone-devel] nvm.experiment.db-compaction
From: |
Markus Schiltknecht |
Subject: |
[Monotone-devel] nvm.experiment.db-compaction |
Date: |
Sun, 23 Mar 2008 17:20:21 +0100 |
User-agent: |
Mozilla-Thunderbird 2.0.0.9 (X11/20080109) |
Hi,
during this easter hack, I've managed to get further with
nvm.e.db-compaction. Rememder? The branch turning all database and
in-memory ids into binary representation. It now passes all unit tests
except schema_migration as well as those which are currently failing on
mainline as well.
On my first attempt, which converted only the database ids, somebody
objected that it would slow monotone down. Therfore I've moved on and
converted all in-memory ids as well.
In the mean time, Zack committed a patch which allows monotone to
recycle the botan pipes, which are used for hex en-/decoding among other
things.
I've done some benchmarking, mainly with
nvm.cbench/examples/annotate.sh. I've been using the following tests,
with 20 repetitions, each.
-b annotate-a='Annotate(ExistingRepo("../monotone.db"),
"f97d974ae4670ce18c438613420d355fd8de80d4", "Makefile.am")' \
-b annotate-b='Annotate(ExistingRepo("../monotone.db"),
"f97d974ae4670ce18c438613420d355fd8de80d4", "paths.hh")' \
-b annotate-c='Annotate(ExistingRepo("../monotone.db"),
"f97d974ae4670ce18c438613420d355fd8de80d4", "xdelta.cc")' \
-b annotate-d='Annotate(ExistingRepo("../monotone.db"),
"f97d974ae4670ce18c438613420d355fd8de80d4", "database.cc")' \
I'm on an Intel EMT64 box, Core 2 Duo, SATA discs.
I've tested the following monotone revisions:
* ref: a reference monotone binary from net.venge.monotone. Rev:
a6e585755b93a0910cca494cd82d060333660ab5
* dbc: from db-compaction branch, with all binary ids, but still uses
L(FL(... encode_hexenc(..))) very often, which isn't conditional in this
revision: 6e24d09551aabc0ca5bc7cda622b1d1eb4b4a76f
* dbc_clog: the conditional logging variant, where the L(FL( macro
features an if (debug... before invoking the format parsing code.
Especially here, that saves quite a bit of time.
Here are the results:
ann-a-ref ann-a-dbc ann-a-dbc_clog p
annotate-avg-resident-MiB 23.36 23.24 23.20 0.63
annotate-avg-size-MiB 119.30 119.68 119.80 0.06
annotate-max-resident-MiB 35.06 34.59 34.44 0.00
annotate-max-size-MiB 132.07 131.56 131.52 0.00
annotate-num-samples 2410.00 2398.05 2292.70 0.22
annotate-system-time 0.86 1.19 1.17 0.00
annotate-user-time 3.67 6.03 5.94 0.00
annotate-wall-time 12.18 12.13 11.60 0.23
ann-b-ref ann-b-dbc ann-b-dbc_clog p
annotate-avg-resident-MiB 12.69 12.48 12.47 0.01
annotate-avg-size-MiB 106.35 106.99 106.88 0.02
annotate-max-resident-MiB 19.78 19.69 19.56 0.00
annotate-max-size-MiB 116.86 116.72 116.75 0.00
annotate-num-samples 762.50 757.50 696.00 0.52
annotate-system-time 0.18 0.20 0.20 0.25
annotate-user-time 0.51 0.79 0.77 0.00
annotate-wall-time 3.87 3.84 3.53 0.52
ann-c-ref ann-c-dbc ann-c-dbc_clog p
annotate-avg-resident-MiB 13.07 13.39 13.19 0.31
annotate-avg-size-MiB 106.41 108.12 108.68 0.12
annotate-max-resident-MiB 19.46 19.54 19.39 0.00
annotate-max-size-MiB 116.60 116.56 116.58 0.00
annotate-num-samples 895.40 1005.50 837.00 0.72
annotate-system-time 0.24 0.29 0.35 0.00
annotate-user-time 0.74 1.21 1.18 0.00
annotate-wall-time 4.56 5.09 4.25 0.72
ann-d-ref ann-d-dbc ann-d-dbc_clog p
annotate-avg-resident-MiB 27.66 27.27 26.97 0.01
annotate-avg-size-MiB 124.15 123.89 123.56 0.26
annotate-max-resident-MiB 36.85 36.44 36.36 0.00
annotate-max-size-MiB 133.90 133.42 133.52 0.00
annotate-num-samples 3531.80 3210.25 3199.20 0.00
annotate-system-time 0.99 1.22 1.19 0.00
annotate-user-time 7.84 9.56 9.42 0.00
annotate-wall-time 17.86 16.24 16.18 0.00
To conclude: speed is somewhat better for annotate (didn't measure
others, yet), while we also see a space saving of ... ehm.. what was it?
8%? Don't remember exactly, at the moment. But the saving has been
confirmed already. ;-)
Now, I'd like to land that branch on mainline as soon as feasible. I
know the encode_hexenc() calls all over the place are annoying, but IMO
we should revamp the vocab macros anyway. Holding of this binary id
branch until then doesn't sound attractive to me.
I'm trying to fix the schema_migration test as soon as possible. That
shouldn't prevent others from reviewing.
A small note: after db migrate, you need a db regenerate_caches, despite
monotone not telling you... To be fixed with schema_migration...
Regards
Markus
- [Monotone-devel] nvm.experiment.db-compaction,
Markus Schiltknecht <=