Re: [Gnu-arch-users] Slow inventories on large source trees

[Aaron Bentley]

Tom Lord wrote:

>     > Is that necessary?  With names tagging, it shouldn't need to stat 
>     > anything, should it?
>
> Yes, it's necessary.   `find' can sometimes get by with just a `chdir'
> that might fail but `inventory' can not.

Surprising to me, but irrelevent, since we're not losing time in stats 
anyhow.  Has anyone tested tla on Reiser4 (where every file is also a 
directory)?

>     > [other message]
>
>     > It's calling filename_matches 89750 times -- about 21 times per file in 
>     > the Wine tree, so I suspect that can be reduced, hopefully to single digits.
>
> You elsewhere mentioned that that's a `changes' profile, not
> `inventory' -- so you're counting _2_ inventories.   The actual
> average is about 10.5 times per file.

For inventory, it was about 7 times per file.  Perhaps there's an extra 
inventory call in "changes" that we can eliminate?

> Abently, are you interested in working on this?  

I'll see how hairy it is.  I'm not expert with regexes, but I'm 
interested.  (Abentley with two e's, please.)

> Do you know about the
> `cut' operator in Rx?  (I.e., you don't want to combine those
> filename_matches calls by adding parentheses to regexps.  You want to
> arrange for the final state label of the dfa to tell you which pattern
> matched.)

Oh, good.  I was afraid we'd lose classification by combining regexes.

I also considered memoizing the regex results.  Obviously, that won't 
help "changes" if the regexes change, but if assoc_tables are faster, it 
might be a win for the common case.

Aaron
--
Aaron Bentley
Director of Technology
Panometrics, Inc.