RE: Make and Clusters

[Cook, Malcolm]

If your cluster is Open Grid Engine, another option is qmake - 
http://gridscheduler.sourceforge.net/htmlman/htmlman1/qmake.html - which 

" is based on gmake (GNU make), version  3.78.1.
" All Sun Grid Engine options valid with  qsub(1)  or  qrsh(1) can  be  
specified with qmake
"The syntax of qmake makefiles corresponds to  gmake  and  is described in the 
"GNU Make Manual".

Alas, to my knowledge this project has not kept up with developments in GNU 
make.  

address@hidden

 > -----Original Message-----
 > From: Help-make [mailto:address@hidden
 > On Behalf Of Paul Smith
 > Sent: Tuesday, October 10, 2017 9:28 AM
 > To: address@hidden
 > Subject: Re: Make and Clusters
 > 
 > On Tue, 2017-10-10 at 15:53 +0200, David Delahaye wrote:
 > > I wonder how the make command is compatible with clusters. I know the
 > > "-j" option, which allows us to run several jobs simultaneously and
 > > uses the several cores of a given processor. But if we have several
 > > nodes of computation of a cluster, is the command "make -j" able to
 > > run over these nodes (transparently for the user)? I know there are
 > > specific commands to run jobs on a cluster, I would like to know how
 > > "make -j" is compatible with these commands.
 > 
 > GNU make doesn't have any built-in support for building on clusters.
 > To do so it would have to accept all sorts of other arguments which
 > defined what machines were in the cluster, what authentication methods
 > are needed to access the cluster, etc. for all different sorts of
 > cluster management software, including plain ssh.  Clearly (from the
 > GNU make manual :)) it doesn't provide any of that.
 > 
 > However, if you have cluster management software you can just change
 > the commands invoked by make to run those commands in a cluster
 > instead.  There are many ways to do it: for one, you can write a
 > wrapper script around your compiler and set the make variable to run
 > that wrapper instead.  Your wrapper script will send the compilation
 > job to the cluster, and stay alive until the job has been completed so
 > it can tell make it's done and what the result was (success/failure).
 > 
 > You can also use other facilities such as distcc http://distcc.org/ to
 > distribute compilation: these are often designed to work well with GNU
 > make's -j option.
 > 
 > _______________________________________________
 > Help-make mailing list
 > address@hidden
 > https://lists.gnu.org/mailman/listinfo/help-make