[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Monotone-devel] Large repo performance (again)

From: Jeff Rizzo
Subject: [Monotone-devel] Large repo performance (again)
Date: Fri, 30 Nov 2012 18:48:54 -0800
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20121026 Thunderbird/16.0.2

I haven't had a chance to attempt an import of the NetBSD repository in a while, but recent[1] successes the NetBSD folks have had cleaning up some of the weirder issues the repo and importing it into fossil and git made me want to give it a try under mtn again. Here's a brief rundown/report of my success.

Monotone version:  1.0:
domU22:riz  /usr/src> mtn --version
monotone 1.0 (base revision: a7c3a1d9de1ba7a62c9dd9efee17252234bb502c)

The CVS repo of "src" is almost 9GB:

domU22:riz  /usr/src> sudo du -sh /cvsroot/src
8.7G    /cvsroot/src

...and it takes approximately 19 hours on a Xen DomU with 10GB RAM and 4 cpus to import it into monotone, yielding a db of 3.0 GB:

mtn -d -b org.NetBSD.src cvs_import /cvsroot/src
{chug, chug ... 19 hours later}

domU22:riz  ~> du -sh

This is great! Much better than a couple years ago. It's actually doable, for one. :-}

Now, as far as actually *using* the repository goes...

domU22:riz  /usr/src> time mtn -d ~/ -b org.NetBSD.src co .
mtn -d ~/ -b org.NetBSD.src co . 50.41s user 867.94s system 26% cpu 57:22.98 total

... nearly an hour to check out!  it's 1.2GB of sources, checked out.

Then, *switching branches* took almost 2h, and failed!

domU22:riz /usr/src> time mtn update -r h:org.NetBSD.src.netbsd-6 --move-conflicting-paths
mtn: expanding selection 'h:org.NetBSD.src.netbsd-6'
mtn: expanded to '115b26019a7cca13433bf1915c5ee025df11134b'
mtn: selected update target 115b26019a7cca13433bf1915c5ee025df11134b
mtn: target revision is not in current branch
mtn: switching to branch 'org.NetBSD.src.netbsd-6'
mtn: [left]  5347af45af8d2944ac167f5ce5cdfe59cc123e9d
mtn: [right] 115b26019a7cca13433bf1915c5ee025df11134b
mtn: warning: cannot drop non-empty directory ''
mtn: moved conflicting path 'OLD' to '_MTN/resolutions/OLD'
mtn: error: renaming 'crypto/external/bsd/openssl/dist/crypto/sha' to '_MTN/detached/314500' failed: Too many links mtn update -r h:org.NetBSD.src.netbsd-6 --move-conflicting-paths 75.34s user 2874.20s system 47% cpu 1:43:52.94 total
domU22:riz  /usr/src>

Clearly, there's still a way to go. If anyone wants to work with me on this (I'm not a C++ guy, though I can do some C), please let me know.


[1] OK, maybe 1.5 years ago.  But it *feels* recent!

reply via email to

[Prev in Thread] Current Thread [Next in Thread]