[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Monotone-devel] Re: slowness of simple operations like "add"

From: graydon hoare
Subject: [Monotone-devel] Re: slowness of simple operations like "add"
Date: Mon, 14 Feb 2005 13:39:26 -0500
User-agent: Mozilla Thunderbird 1.0 (X11/20041206)

Georg-W. Koltermann wrote:

     1. It seems 51% of the time is spent in calculate_base_manifest(),
        index [8] in the profiling data.  Why do we need to calculate
        the base manifest at all?  Couldn't we cache that in MT/revision
or some other file?

we could, but we don't. I prefer not to have extra files in the MT directory, since they must be cross-checked for consistency. probably it'll be sufficient to store the revision->manifest mapping in a separate table, rather than re-reading the revision each time we want to look that up.

     2. Most time of the above is spent in parse_edge() [19] and
        parse_change_set() [29].  Why does that take so long when I have
        only ONE revision in the database???

the single revision you have is large: it involved 5000 additions. unfortunately our current code involves *reading* the base revision (the one before the one you're committing) in order to work out the base manifest. this is unnecessary, but it's what we currently do. we could change this to speed it up, as I mention above, with a separate table.

     3. Then 43% of time is spent in build_addition() [15] with major
        contributions by apply_path_rearrangement() [25] and
        concatenate_rearrangements() [20].  This is an area where I
        cannot even guess what it does.

well, we have to check that what you're trying to do makes sense. to do that we must compare your request (adding a file) to the existing manifest, subject to any other not-yet-committed rearrangement stored in the MT/work file. to do this we build a path set representing your manifest, then rearrange it under the existing MT/work rearrangement, then use that rearranged path set to check the validity of your request. this involves, among other things, an intermediate "scratch revision" which adds all 5000 files in your manifest to a rearrangement.

there are other ways we could check this which involve fewer, and less expensive, intermediate forms. it is also possible to modify the internal consistency checking of rearrangements, such that they are cheaper to check, so the intermediate forms don't cost as much. there are a few things we could try.

Ideally I would think that the "monotone add" could be just a quick
check in the database if the file is already present and then a write to
the work file.  But maybe I am too naive here.

that is, in broad brush strokes, what it's doing. unfortunately you have a lot of files in your manifest, and a lot of additions in your base revision, and both those turn into heavy factors on the intermediate things we need to calculate. clearly we need to work on making those things cheaper. thanks for the feedback.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]