[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: Make and Clusters
RE: Make and Clusters
Tue, 10 Oct 2017 14:44:20 +0000
If your cluster is Open Grid Engine, another option is qmake -
http://gridscheduler.sourceforge.net/htmlman/htmlman1/qmake.html - which
" is based on gmake (GNU make), version 3.78.1.
" All Sun Grid Engine options valid with qsub(1) or qrsh(1) can be
specified with qmake
"The syntax of qmake makefiles corresponds to gmake and is described in the
"GNU Make Manual".
Alas, to my knowledge this project has not kept up with developments in GNU
> -----Original Message-----
> From: Help-make [mailto:address@hidden
> On Behalf Of Paul Smith
> Sent: Tuesday, October 10, 2017 9:28 AM
> To: address@hidden
> Subject: Re: Make and Clusters
> On Tue, 2017-10-10 at 15:53 +0200, David Delahaye wrote:
> > I wonder how the make command is compatible with clusters. I know the
> > "-j" option, which allows us to run several jobs simultaneously and
> > uses the several cores of a given processor. But if we have several
> > nodes of computation of a cluster, is the command "make -j" able to
> > run over these nodes (transparently for the user)? I know there are
> > specific commands to run jobs on a cluster, I would like to know how
> > "make -j" is compatible with these commands.
> GNU make doesn't have any built-in support for building on clusters.
> To do so it would have to accept all sorts of other arguments which
> defined what machines were in the cluster, what authentication methods
> are needed to access the cluster, etc. for all different sorts of
> cluster management software, including plain ssh. Clearly (from the
> GNU make manual :)) it doesn't provide any of that.
> However, if you have cluster management software you can just change
> the commands invoked by make to run those commands in a cluster
> instead. There are many ways to do it: for one, you can write a
> wrapper script around your compiler and set the make variable to run
> that wrapper instead. Your wrapper script will send the compilation
> job to the cluster, and stay alive until the job has been completed so
> it can tell make it's done and what the result was (success/failure).
> You can also use other facilities such as distcc http://distcc.org/ to
> distribute compilation: these are often designed to work well with GNU
> make's -j option.
> Help-make mailing list