Re: [Monotone-devel] Re: why is monotone so slow?

[Georg-W. Koltermann]

Am Di, den 14.09.2004 schrieb Bruce Stephens um 15:51:

> In this particular case (with a largish source tree), I'd guess the
> big cost is reading all the files and doing SHA1 on them.  I remember
> when this kind of problem hit Arch, and (eventually) code was put in
> to remember mtime of checked out paths, and that means you just need
> to do a stat, which turned out to be quicker (than doing a diff,
> IIRC).
> 
> As far as I can see, monotone doesn't do that, so it doesn't have a
> way to tell that a file is untouched except by calculating its hash.
> I'm not suggesting that's a bad way to do things, but (if I'm right
> about this) it's going to cause a performance problem on large trees.
> 
> It's the kind of thing I wouldn't build into a system to begin
> with---it's not information that you'd stick into the repository, so
> it should be an easyish optimisation whenever someone is bothered
> enough to add it.

If you look at my time(1) output in the original posting you can see
that only 8.6 s out of the 42 s were user CPU.  So however fancy you get
with compiler optimization levels, remembering modification times, and
all that you will at most save 8.6 s :(

The system CPU consumption was 1.2 s, even a little less than for OpenCM
(1.5 s).  I take this as an indication that monotone issues less
filesystem related calls.  Well I'm guessing here but the only source of
system CPU consuption that I see is I/O calls, and of those the
filesystem traversal calls are by far the most expensive.  Unless you do
unbuffered I/O with small request sizes, which I doubt anyone will do
nowadays.  And given that OpenCM stores the repository as individual
files, all indications fit nicely.  This is only a side observation
anyway.

So in summary I am confident that what I see as delay is I/O wait time. 
And since the I/O to the workspace should be the same for all CM
systems, I suspect differences in the I/O to the repository.  For
monotone this is database I/O, hence my question for optimizing the SQL
settings (read ahead? DB page size? DB buffer pool size?).

--
Regards,
Georg.