Re: help! Octave, connecting laptops to "supercomputers"

[Pedro]

On Sun, Feb 3, 2013 at 3:33 PM, Pedro <address@hidden> wrote:
> On Sun, Feb 3, 2013 at 12:19 PM, Thomas Weber <address@hidden> wrote:
>>
>> Hi,
>>
>> reading through your mails gives me the impression that you have no idea
>> what you actually want, sorry.
>
> It doesn't matter ;)
> I think that every response is giving interesting information about
> that field. And helps me, and I hope also other people that is having
> the same problems.
>
>> If you process data on a host and want to look at a graphics from this
>> data processing, then this image must be sent to your local machine. It
>> doesn't matter at all what kind of connection you use; the image must be
>> sent to your machine. If you need to sync lots of data (like > 10 MB)
>> over a slow connection, then rsync is the best general purpose solution.
>>
>> Yes, there are options that are better then forwarding an X connection
>> to your laptop via ssh. But then it depends on what you want. Because,
>> if you just want to run a script via the Octave command line, then you
>> wouldn't even forward your X session, but just run Octave inside GNU
>> screen or tmux.
>
> Yes, I see screen or tmux really useful for keep running processing
> when you disconnect session with ssh [1] [2]
>
>> On Thu, Jan 31, 2013 at 02:00:33AM +0100, Pedro wrote:
>> > First of all, Writing this mail I wanted to know what use octavian
>> > people like me (or not) when need to work with hard processes. 4/4 for
>> > going in "ssh -X", well, I have to re-think on that!
>> >
>> > I said no to "ssh -X" because I'm looking for alternatives on
>> > connecting with computer/computers that process the data.
>> >
>> > - I've never used. (read last line of this mail)
>> >
>> > - from ssh -X I read that is innerently insecure, because run a root
>> > process: the X server, read this (old) review about this [1].
>> >
>> > - Also, because is slow (or inefficient), that's because is encrypting
>> > all the communications to the server.
>> > But I only want to "pass the code strictly necessary".
>> > Because if I use the ssh connection as a workstation it's excessive
>> > for what I want.
>> > Imagine looking images, sound or video with a ssh connection in a
>> > "streaming way".
>> > I could do rsync? ... Yes, but are things that take you off your
>> > concentration, no? It means, that make you thing on two machines
>> > constantly.
>>
>> I suggest you actually start with ssh and rsync and then solve problems
>> as they come along.
>>
>>
>> > I would like to imagine "one machine", could be with virtualization?
>> > cloud computing? "p2p processing" ? Assymetric processing?
>>
>> Your problem definition is so broad that even "pocket calculator" seems
>> to be in the solution set. What do you actually want to do?
>
> I wanted to know what are doing other people with the problem of "big
> calculations", and discuss different methods.
> And I put that keywords for inspire people to post a solution in that way.
>
>> > I don't understand a lot how it works distcc, but see this:
>> > compilation of kernel distributed: the guy is in his computer, and
>> > divide tasks to others, after it, they have his output file. [1]
>> > Compiling means that a personalized task (hardware, processor
>> > architecture...) but delivered to process by others
>>
>> distcc is a specialized tool for a very specific problem. In other
>> words, it is the complete opposite of your approach.
>>
>> > For "ssh -X" users, could they give some tricks to make life easier?
>> Depends; what problems do you have?
>>
>> For the record, I used to have Octave running on a 40 node cluster
>> several years back. One Octave session was started on each host insided
>> a screen session with 40 windows. Data was put onto the central master
>> node,
>> home directories were NFS-mounted into each node. Numeric results were
>> saved on each node into the shared home directory and finally images
>> from this data was generated on the master node. The only data that I
>> then downloaded from this system was the images. This worked reasonably
>> well for me. If your problem is different, then your solution will be
>> different.
>>
>> All of this only makes sense if your calculations takes some time. If
>> you have to wait like 1 minute, then the time to set up your specific
>> solution will be much larger than the time for the calculations on your
>> small machine.
>
> I appreciate your sharing.
> And I have a question:
> How you manage this 40 screens (nodes)? In every screen you put
> different commands/scripts (by hand), to have the processing
> distributed?
> Or what tool / interface you use to do it?
>
> Thanks!
> Pedro
>
> [1] 
> http://askubuntu.com/questions/8653/how-to-keep-processes-running-after-ending-ssh-session
> [2] 
> http://serverfault.com/questions/19634/how-to-reconnect-to-a-disconnected-ssh-session

Hey guys, given the comments of running via ssh. Perhaps this is the
best way to start: install a simply octave on a SSI system. That's why
I was looking for.

Also I add some solutions (Live CD/USB). I found it thanks to the
keyword beowulf and his lists (take a look to beowulf cluster concept
[1] !)

easy to go
    Live CD/USB
        http://pareto.uab.es/mcreel/PelicanHPC/
        http://birg1.fbb.utm.my/birghpc/
        http://birg1.fbb.utm.my/birghpcc/
            for cuda
    http://www.kerrighed.org/wiki/index.php/Main_Page
    OpenSSI
        http://openssi.org/cgi-bin/view?page=openssi.html
        https://en.wikipedia.org/wiki/OpenSSI

[1] https://en.wikipedia.org/wiki/Beowulf_cluster