[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Monotone-devel] Re: slowness of simple operations like "add"
From: |
graydon hoare |
Subject: |
[Monotone-devel] Re: slowness of simple operations like "add" |
Date: |
Mon, 14 Feb 2005 13:39:26 -0500 |
User-agent: |
Mozilla Thunderbird 1.0 (X11/20041206) |
Georg-W. Koltermann wrote:
1. It seems 51% of the time is spent in calculate_base_manifest(),
index [8] in the profiling data. Why do we need to calculate
the base manifest at all? Couldn't we cache that in MT/revision
or some other file?
we could, but we don't. I prefer not to have extra files in the MT
directory, since they must be cross-checked for consistency. probably
it'll be sufficient to store the revision->manifest mapping in a
separate table, rather than re-reading the revision each time we want to
look that up.
2. Most time of the above is spent in parse_edge() [19] and
parse_change_set() [29]. Why does that take so long when I have
only ONE revision in the database???
the single revision you have is large: it involved 5000 additions.
unfortunately our current code involves *reading* the base revision (the
one before the one you're committing) in order to work out the base
manifest. this is unnecessary, but it's what we currently do. we could
change this to speed it up, as I mention above, with a separate table.
3. Then 43% of time is spent in build_addition() [15] with major
contributions by apply_path_rearrangement() [25] and
concatenate_rearrangements() [20]. This is an area where I
cannot even guess what it does.
well, we have to check that what you're trying to do makes sense. to do
that we must compare your request (adding a file) to the existing
manifest, subject to any other not-yet-committed rearrangement stored in
the MT/work file. to do this we build a path set representing your
manifest, then rearrange it under the existing MT/work rearrangement,
then use that rearranged path set to check the validity of your request.
this involves, among other things, an intermediate "scratch revision"
which adds all 5000 files in your manifest to a rearrangement.
there are other ways we could check this which involve fewer, and less
expensive, intermediate forms. it is also possible to modify the
internal consistency checking of rearrangements, such that they are
cheaper to check, so the intermediate forms don't cost as much. there
are a few things we could try.
Ideally I would think that the "monotone add" could be just a quick
check in the database if the file is already present and then a write to
the work file. But maybe I am too naive here.
that is, in broad brush strokes, what it's doing. unfortunately you have
a lot of files in your manifest, and a lot of additions in your base
revision, and both those turn into heavy factors on the intermediate
things we need to calculate. clearly we need to work on making those
things cheaper. thanks for the feedback.
-graydon