EC2 integration for ssh hosts?

[Matt Oates (Home)]

Hi All,

To start out thanks for the great work on parallel it lets me get my
work done! Stuff that took a day now takes an hour :)

I'm a bioinformatician and I need to deal with a tonne of data in
parallel. GNU parallel has been great for this on our 24 CPU machine
so far! Looking at the archives for this list, I'm looking forward to
--pipe with bated breath. FASTA files are all I deal with.

However, we're reaching the limits of most hardware laying around, my
research group are moving to the Amazon Elastic Cloud. I was wondering
if anyone can gauge how much heartache there would be in hacking on
GNU parallel using the Net::Amazon::EC2 module to bring up a bunch of
cloud instances, and then pass this off as the list of ssh remote
hosts? I could just wrap something around parallel in the shell but it
feels like a nice optional feature for others..?

Best Wishes,
Matt Oates