Re: CPU-specific builds

[Ben Woodcroft]

Thanks for the responses Ludo and Eric.


On 02/10/16 23:33, Ludovic Courtès wrote:
> > Currently, we build a single set of x86_64 packages assuming SSE but
> > not SSE2 instructions, but sometimes it would be nice to use more
> > recent instructions like AVX.
> Isn’t SSE2 part of the x86_64 base spec?  I always forget.
I don't think so. I always refer to Mark's comment when I forget:
https://lists.gnu.org/archive/html/guix-devel/2016-07/msg01534.html

> > So I'm wondering if there is some way to specify a system more
> > specific than 'X86_64'? I tried simply adding '--with-arch=haswell' as
> > a configure argument in gcc-4.9 so that flag became the default for
> > gcc usage and saw some performance improvements, though I did have to
> > disable tests in gnutls.
> Do you have performance figures for some CPU-intensive applications?
>
> What software are you most interested in?
We tend to run software whose runtime is dependent on the input data, so 
it is hard to say. But up to days or weeks of walltime in some cases. It 
takes a lot of power researching climate change..

As an anecdote, adding "-march=haswell" shaved 13% off the runtime of 
diamond, ~20% if the CPUs were contended.

> > Hardcoding that configure flag is definitely less than ideal, I'm
> > wondering if there is some better way that would enable us to share
> > package updates and even substitutes for these systems? My initial
> > thought is extending the triplet 'x86_64-unknown-linux-gnu' somehow,
> > but I suspect others have better ideas?
> I’m not sure how to do this.  Having, say, an x86_64avx-linux system
> type (not triplet) would be impractical because it would be entirely
> separate from x86_64-linux (different derivations).
Yes my original plan was to rebuild all packages locally, but that was 
probably too optimistic as I reckon there will be too much work involved 
in maintaining it.

> Ideally, software for which using these CPU extensions makes a
> significant difference would do what glibc does, which is to provide
> several implementations of the relevant functions (one for SSE2, one for
> AVX, etc.) and have the right one be selected at load time via an IFUNC
> or similar mechanism.
That sounds useful in some cases, but it is probably too much of a 
stretch for most bioinformatics packages.

In the end I think I'll just compile the specific packages we are 
specifically interested in. I attached some example code in case anyone 
is interested. But this brought up a few questions:

1) I also tried using --expression e.g. guix build --expression '(@@ (my 
packages cpu-specific) diamond-cpu-specific)' but that fails to compile 
as if the GUIX_PACKAGE_PATH is ignored, is that unexpected?

2) Is something amiss with gcc-toolchain-6? Compiling with it, diamond 
complains of a missing stdlib.h.

Thanks,
ben

cpu-specific.scm
Description: Text Data