help-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Large non-recursive make not scaling. Patch.


From: Paul Smith
Subject: Re: Large non-recursive make not scaling. Patch.
Date: Tue, 23 Mar 2010 14:43:43 -0400

On Tue, 2010-03-23 at 00:41 -0700, Corey Brenner wrote:
> I achieved both, but ran into a problem of the main variable hash not
> scaling well when setting up variable "namespaces" (in reality, a
> large variable name, including relative path,) when the build went
> past a couple of thousand directories.
> 
> I have implemented a PATRICIA trie for GNU Make, to take care of the
> problem caused by linear probing in the global hash.  I have attached
> patches against the 3.81 release, and tonight's CVS.

Hi Corey; thanks for this work.  Can I just be clear: the problem you're
trying to solve is that if variable names themselves are very large then
computation of the hash alone can add considerable time to the lookup,
is that right?  That's what I assume from the fact that you're using a
PATRICIA trie.

I would love to see some kind of comparison of time/space tradeoffs
(some people have LOTS AND LOTS of variables, and I wonder what the
storage efficiency is for the trie vs. the simple open hash).  Doesn't
have to be anything formal but if you had some runs, maybe using
valgrind or even just GNU libc's heap statistics functions, that showed
memory usage differences for larger projects, and/or performance
differences, that would be very nice!





reply via email to

[Prev in Thread] Current Thread [Next in Thread]