Re: [Monotone-devel] Improving client side performance of pulls

[Nathaniel Smith]

On Wed, Aug 02, 2006 at 01:14:43AM -0700, Eric Anderson wrote:
> I was surprised there was so much time spent in lock and unlock, and
> by sampling some back traces in gdb, I found it seemed to be all
> memory allocation.

Recent versions of oprofile have the "-c" switch, which means that it
collects stack trace samples.  Usage is basically "opcontrol -c 100"
(or <100 if you want it to only look at the top few stack frames),
then "opreport -c", and see
  
http://colabti.de/irclogger/irclogger_log/monotone?date=2006-02-28,Tue&sel=23#l36
for interpreting the output (linked from
  http://venge.net/monotone/wiki/PerformanceWork
).

> I see a number of possible solutions:
>   1) write a custom string allocation system that runs without locks and 
>      modify all of the string stuff to use that.  Very ugly, but possibly
>      made easier by the use of the vocab wrappers.

I wonder if it is... if so that's great, but custom allocation stuff
is tricky, because it changes the type of every STL object you set a
custom allocator in, so every time you mention the type of such an
object you have to mention the allocator as well.  typedef's can help,
but I worry a bit about all the places we say "std::string"...

>   2) include a non-unicode version of boost-regex in monotone so that it 
>      can be built without threading support. Ugly duplication of code and
>      build bits, and doesn't fix the problem unless all the vendors ship
>      non-pthread versions of the other boost libraries.

Unicode support is actually kind of a nice thing to have, given that
we are supposed to be all i18nized and stuff :-).  I don't know that
we actually _use_ whatever boost-regex calls "unicode support", though
(probably wide-string support, which we certainly don't use).

>   3) write a fake shim library that emulates all of the pthread calls, but
>      does nothing and aborts if something tries to create a thread. This
>      just seems wrong as an approach, but is possibly the simplest to 
>      implement.

Do we know if the locking is done by the STL allocators, or by
malloc/new themselves?  Because if it's the later, doesn't C++ provide
a hook to define a global replacement for operator new?  And
the same machinery that makes quick_alloc.h work must know how to
access some sort of low-level non-locking operator-new primitives?

-- Nathaniel

-- 
In mathematics, it's not enough to read the words
you have to hear the music