[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] git-download: Speed up 'git-predicate'.

From: Christopher Baines
Subject: Re: [PATCH] git-download: Speed up 'git-predicate'.
Date: Fri, 2 Jun 2017 08:34:59 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0

To provide some anecdotal evidence on the difference this makes, I wrote
a little test script [1]. It should roughly follow how git-predicate is
used with the local-file gexp.

Running the current implementation of git-predicate on the guix
repository with 50 trials takes:

  real    0m5.374s
  user    0m4.052s
  sys     0m1.484s

Compared to the following for the patched implementation:

  real    0m8.398s
  user    0m5.040s
  sys     0m2.016s

This shows that the patched implementation has decreased the
performance. Using the real value, the test time has increased by ~3
seconds. This corresponds roughly to a change from ~0.11 seconds
previously, to ~0.17 seconds for each individual traversal of the entire

I encountered a performance issue when trying to use git-predicate with
this repository [2], which I have written a Guix package for here [3].
Instead of the 1463 files that git ls-files reports locally for the guix
repository, smart-answers contains 26732 files.

The running time with smart-answers is left as an exercise to the
reader, all I can say is that on my machine, it takes more than 40
minutes for just 1 trial.

Using this patch, running the test script with 1 trial gives:

  real    0m4.917s
  user    0m3.640s
  sys     0m1.428s


(use-modules (srfi srfi-1)
             (srfi srfi-26)
             (ice-9 ftw)
             (guix git-download))

(define (test file select?)
  (let dump ((f file) (s (lstat file)))
    (case (stat:type s)
       (display "."))
        (lambda (e)
          (let* ((f (string-append f "/" e))
                 (s (lstat f)))
            (if (select? f s)
                (dump f s))))
        (scandir f (negate (cut member <> '("." ".."))) string<?)))
       (display ".")))))

(define (run-test dir trials)
  (chdir dir)
   (lambda (trial)
     (test dir (git-predicate dir)))
   (iota trials)))

(run-test (second (command-line))
          (string->number (third (command-line))))


Attachment: signature.asc
Description: OpenPGP digital signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]