[bug #38092] Optionally support distributed memory machines using MPI

[anonymous]

Follow-up Comment #14, bug #38092 (project findutils):

> Even if there was, you would have to pass this "N" to xargs,
> and that would have to be done by the caller - mpirun.

Correct although technically N isn't given by mpirun but by the MPI library
environment in which a program is executed (typically by mpirun or aprun or
some such program).

The only way to know what N is in this environment is to call MPI_* functions.
But once I see an example of stock xargs only processing every #define N 3 -th
argument it should be easy to adjust N at runtime with MPI instead.

> The problem is that you would like to have xargs to process
> every Nth argument on node A while xargs on node B
> should process every (N+1)th argument etc. 

Yes and as far as I know there is no way (standard compliant, portable, ...)
to find out which xargs should process which files without calling MPI
functions. That's just the nature of the environment.

> it is the job of rank 0 to distribute the data to the others.

Not if each process can find out that information on its own. In all systems
that I've tried all processes have the same view of the filesystem (lustre.org
seems to be a popular implementation) and so can open and read the same
argument file. They just have to know which items to process and which ones to
skip.

> I think you're better off asking in the OpenMPI forum.

Using MPI isn't the problem here for me but figuring out how to make xargs.c
only process e.g. every Nth item in an argument file is. Modifying any of the
loops in main() didn't seem to accomplish this. Once xargs can skip items it
should be very easy to calculate the items to skip using MPI instead.

> the MPI stuff is usually linked dynamically into a program

In this respect MPI is the same as any other library. On some systems the
parallel executable must be static but in e.g. on my ubuntu laptop it is
dynamic. And anyway if MPI portions are behind #defines non-MPI systems
shouldn't be affected in any way.
Then the regular xargs could be compiled as usual and a configure option could
be provided to also compile an MPI aware version with for example the name
mpixargs.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?38092>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/