[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

speed up gnulib-tool a bit

From: Ralf Wildenhues
Subject: speed up gnulib-tool a bit
Date: Fri, 15 Sep 2006 18:47:56 +0200
User-agent: Mutt/1.5.13 (2006-09-01)

Hello Bruno, all,

For some invocations, gnulib-tool is rather slow[1], which impacts
productivity a bit.  The following tries to improve things without
sacrificing portability, while hopefully it doesn't compromise
readability.  ;-)

First, `sort -u' is already used in gnulib-tool, so it would seem only
consistent to drop `uniq' where possible, even as it does not lead to a
noticeable speedup.

Then, the transitive closure algorithm need not walk any module names in
the dependency graph more than once.  (Looking up and checking a module
is rather expensive due to the file operations and fork&exec involved.)

Also, the rewriting of file names may be done in batch.  Note that this
may cause the loop that calls `func_add_or_update' to be executed in a
subshell, but this shouldn't matter: the function does not change any
state.  (Should I provide a similar change for func_create_testdir?)

And the include directives need only be read once.

What do you think?


[1] Times taken on a modern system with:
  time gnulib-tool --import acl alloca argmatch assert backupfile \
    base64 c-strtold calloc canon-host canonicalize chown config-h \
    closeout cycle-check d-ino d-type diacrit dirfd dirname error \
    euidaccess exclude exitfail fcntl fcntl-safer fdl fileblocks  \
    filemode filenamecat fnmatch-gnu fprintftime fsusage ftruncate \
    fts getdate getgroups getline getloadavg getndelim2 getopt \
    getpagesize gettext gettime gettimeofday getugroups getusershell \
    group-member hard-locale hash hash-pjw host-os human idcache \
    inttostr inttypes lchmod lchown lib-ignore

43.07user 25.70system 1:00.87elapsed 112%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+5660467minor)pagefaults 0swaps

18.70user 9.34system 0:26.49elapsed 105%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1995848minor)pagefaults 0swaps

PS: For the bootstrapping of gettext, it saves roughly 40s.

PPS: should the number of files used by a project exceed a few hundred,
it may make sense to exploit bash3's `+=' with a func_append, in order
to avoid the quadratic scaling associated with string enlargement:

Attachment: sort-u.diff
Description: Text document

Attachment: transitive.diff
Description: Text document

Attachment: files-rewrite.diff
Description: Text document

Attachment: includes.diff
Description: Text document

reply via email to

[Prev in Thread] Current Thread [Next in Thread]