[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] build: pull: Compile .scm files in one process.
From: |
Ludovic Courtès |
Subject: |
Re: [PATCH] build: pull: Compile .scm files in one process. |
Date: |
Thu, 12 Nov 2015 21:44:38 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) |
address@hidden (Taylan Ulrich "Bayırlı/Kammer") skribis:
> With that and your further help on IRC, I ended up with the following
> patch.
>
> This takes about 2 minutes on my system. The CPU gets fried during the
> parallel compilation, as expected. ;-)
Nice. :-)
> From 0eaf7d4c264dc531718a4c0b933323f48ea91930 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
> <address@hidden>
> Date: Thu, 5 Nov 2015 23:43:20 +0100
> Subject: [PATCH] build: pull: Compile .scm files in one process.
>
> * guix/build/pull.scm (call-with-process): Removed procedure.
> (report-build-progress): Removed procedure.
> (p-for-each): Removed procedure.
Rather:
… (call-with-process, report-build-progress, p-for-each): Remove.
> + (let ((mutex (make-mutex))
> + (completed 0))
> + (par-for-each
> + (lambda (file)
> + (with-mutex mutex
> + (display #\cr log-port)
> + (format log-port "compiling...\t~5,1f% of ~d files" ;FIXME:
> i18n
> + (* 100. (/ completed total)) total)
> + (force-output log-port)
> + (format debug-port "~%compiling '~a'...~%" file))
> + (let ((go (string-append (string-drop-right file 4) ".go")))
> + (compile-file file
> + #:output-file go
> + #:opts %auto-compilation-options))
> + (with-mutex mutex
> + (set! completed (+ 1 completed))))
> + files)))))
Does it actually work reliably? :-)
I think we’re in trouble if DEBUG-PORT is a real port because ports
aren’t thread-safe in 2.0, and ‘compile-file’ might write to it. Void
ports seem to be OK because their ‘write’ method doesn’t do anything.
So I think we have to do things sequentially when DEBUG-PORT is
non-void.
I’m also concerned about modules accessed by concurrent calls to
‘compile-file’ because modules and hash tables aren’t thread-safe
either.
WDYT?
Thanks to the two of you! Sounds like we’ll soon have a less slow or
even a fast ‘guix pull’. ;-)
Ludo’.
- Re: [PATCH] build: pull: Compile .scm files in one process., (continued)
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/06
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/07
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/10
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/11
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/11
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/12
- Re: [PATCH] build: pull: Compile .scm files in one process., Andy Wingo, 2015/11/12
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/12
- Re: [PATCH] build: pull: Compile .scm files in one process., Andy Wingo, 2015/11/12
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/12
- Re: [PATCH] build: pull: Compile .scm files in one process.,
Ludovic Courtès <=
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/13
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/14
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/26
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/27
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/27
- Re: [PATCH] build: pull: Compile .scm files in one process., Taylan Ulrich Bayırlı/Kammer, 2015/11/27
- Re: [PATCH] build: pull: Compile .scm files in one process., Ludovic Courtès, 2015/11/30
Re: [PATCH] build: pull: Compile .scm files in one process., Andy Wingo, 2015/11/06