[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: CPU-specific builds
From: |
Ludovic Courtès |
Subject: |
Re: CPU-specific builds |
Date: |
Wed, 12 Oct 2016 14:22:16 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) |
Ben Woodcroft <address@hidden> skribis:
> On 02/10/16 23:33, Ludovic Courtès wrote:
[...]
>>> So I'm wondering if there is some way to specify a system more
>>> specific than 'X86_64'? I tried simply adding '--with-arch=haswell' as
>>> a configure argument in gcc-4.9 so that flag became the default for
>>> gcc usage and saw some performance improvements, though I did have to
>>> disable tests in gnutls.
>> Do you have performance figures for some CPU-intensive applications?
>>
>> What software are you most interested in?
> We tend to run software whose runtime is dependent on the input data,
> so it is hard to say. But up to days or weeks of walltime in some
> cases. It takes a lot of power researching climate change..
>
> As an anecdote, adding "-march=haswell" shaved 13% off the runtime of
> diamond, ~20% if the CPUs were contended.
That’s more than I expected. It’d be useful to have a way to specify
this.
>> Ideally, software for which using these CPU extensions makes a
>> significant difference would do what glibc does, which is to provide
>> several implementations of the relevant functions (one for SSE2, one for
>> AVX, etc.) and have the right one be selected at load time via an IFUNC
>> or similar mechanism.
> That sounds useful in some cases, but it is probably too much of a
> stretch for most bioinformatics packages.
Yeah.
> In the end I think I'll just compile the specific packages we are
> specifically interested in. I attached some example code in case
> anyone is interested. But this brought up a few questions:
>
> 1) I also tried using --expression e.g. guix build --expression '(@@
> (my packages cpu-specific) diamond-cpu-specific)' but that fails to
> compile as if the GUIX_PACKAGE_PATH is ignored, is that unexpected?
What error did you get?
> 2) Is something amiss with gcc-toolchain-6? Compiling with it, diamond
> complains of a missing stdlib.h.
Everything’s fine AFAICS. However, note that ‘gnu-build-system’ pulls
in GCC 4.9, glibc, etc. If you add ‘gcc-toolchain’ to the inputs, that
surely conflicts, but I’m unsure which one “wins”; could you check the
‘environment-variables’ file in a build tree?
> (define-public gcc-cpu-specific
> (let ((base gcc-5)) ; gcc-6 does not seem to work.
> (package
> (inherit base)
> (name "gcc-cpu-specific")
> (arguments
> (substitute-keyword-arguments (package-arguments base)
> ((#:configure-flags configure-flags)
> `(append ,configure-flags
> (list (string-append
> "--with-arch=" ,cpu)))))))))
>
> (define-public (cpu-specific-package base-package)
> (package
> (inherit base-package)
> (name (package-name base-package))
> ;; We must set a higher package version so this package is used instead of
> ;; the package in Guix proper.
> (version (string-append (package-version base-package) "-cpu-specific"))
> (inputs
> `(,@(package-inputs base-package)
> ("gcc" ,((@@ (gnu packages commencement)
> gcc-toolchain) gcc-cpu-specific))))))
>
> (define-public diamond-cpu-specific (cpu-specific-package diamond))
> (define-public fasttree-cpu-specific (cpu-specific-package fasttree))
> (define-public blast+-cpu-specific (cpu-specific-package blast+))
> (define-public bwa-cpu-specific (cpu-specific-package bwa))
> (define-public metabat-cpu-specific (cpu-specific-package metabat))
Looks like the right way to start.
We should start thinking about what the ideal interface would look like,
though. At the highest level, I imagine we’d want something like:
guix package --tune=haswell -i diamond
Under the hood that could work by having an arbitrary list of key/value
options passed to build systems, rather than just the system type and
cross-compilation target as is currently the case.
I once proposed something quite similar, but in the context of providing
something equivalent to Gentoo “USE flags”:
https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00177.html
Food for thought…
Ludo’.