Re: parallelizing more actions

[Chris Marusich]

Efraim Flashner <address@hidden> writes:

> While rebuilding glibc-final on aarch64 I realized that the 'strip
> phase took 235 seconds. The relevant code for 'strip from
> gnu-build-system is in guix/build/gnu-build-system.scm, starting at
> line 340, with the actual stripping starting at 398. When I changed
> 'for-each' to 'par-for-each' the time dropped from 235 seconds to 215,
> about an 8.5% savings. I'm pretty sure most of that time was spent
> failing to strip certain files, but it is still a savings.

Nice!

>
> I'm guessing there are other parts that can be parallelized in a
> similar manner.
>
> As for par-for-each (as many threads as cores on the machine) vs
> n-par-for-each (n threads), I think it would be better to use
> n-par-for-each, but I didn't quickly see a way to limit based on the
> number of cores offered to each builder.

I think you want parallel-job-count in (guix build utils):

(define parallel-job-count
  ;; Number of processes to be passed next to GNU Make's `-j' argument.
  (make-parameter
   (match (getenv "NIX_BUILD_CORES")              ;set by the daemon
     (#f  1)
     ("0" (current-processor-count))
     (x   (or (string->number x) 1)))))

-- 
Chris

signature.asc
Description: PGP signature