Introduction About Site Map

RSS 2 Feed RSS 2 Feed

Main Page | Blog Index

Monday, January 23rd, 2012, 3:03 pm

Running Experiments Over SSH

BAD habits die hard. Good habits stay, so over time we do things more effectively. Today’s post may be relevant to post-doctoral folks whose work involves a lot of computation, including over-night or multi-day experiments.

6 or 7 years ago I wrote detailed posts about how I was using KDE to run experiments on entire clusters of Fedora boxes, over SSH. It has been a long time since then and nowadays, rather than use many dual-core boxes I just mostly use a pair of 8-core computational servers. The tricks are mostly the same and the scripts are largely reused from those that I wrote around 2004 (they are publicly available in this Web site). The procedures are mostly the same, but technically, a few things have changed and this post will detail them (aside from that, back then I did my Ph.D. and now I get paid to run these experiments).

In case of network issues, it is important to run everything but the computation locally. This means that code editing, for example, should be done locally. This reduces cursor/menu lag and prevents loss of work in case of loss of connection. It also assures that files get written to more than one place (local plus remote). Kate with KIO (in Kate) use slaves to enable editing of files over SFTP or SCP/SSH, so this functionality ought to be exploited for controlling experiments remotely. For other tasks, separate terminal windows (e.g. Konsole) should be opened, preferably with endless scrolling buffer, for each remote machine. GUIs can be created to enable quick adjustment and running of experiments. A good terminal will stay active and visible even when a remote connection gets closed, in which case, results can still be observed. In addition, it may help to have another terminal window connected to each remote machine in order to track load throughout runtime, as well as other things (one busy session may permit nothing else to be done on the same terminal). Here it is illustrated graphically, based on screenshots just taken.

Full screen view on workspace 8 (research):

Full screen

Let’s break it down into the left and right monitors:

Full screen left

This left side of the dual-head display contains the tabbed-style view of program files that need editing. All those files are using KIO to essentially be seen as local even though editing them modifies them on the remote servers that I connect to (see terminals on the right). In addition, I used Java framework to create a GUI front end for the experiments. It is singleton for each server.

On the other screen I have this:

Full screen right

Shown at the top left is a window that’s invoked by the program when results are ready. By using KDE’s window-specific settings I can force all such windows to always open in workspace 8 (research), so even if I am busy with other tasks the windows will quietly show up where it belongs, sometimes along with dozen other windows that need attention later. On the right side there are terminals connected to the computational servers. One currently gets used to rsync the code across across servers and the other is tracking server loads.

So, this is pretty much how I use workspace 8. I previously explained how I use workspace 2 (there are 12 in total). KDE makes it easy to multi-task without getting distracted.

Technical Notes About Comments

Comments may include corrections, additions, citations, expressions of consent or even disagreements. They should preferably remain on topic.

Moderation: All genuine comments will be added. If your comment does not appear immediately (a rarity), it awaits moderation as it contained a sensitive word or a URI.

Trackbacks: The URI to TrackBack this entry is:

Syndication: RSS feed for comments on this post RSS 2

    See also: What are feeds?, Local Feeds

Comments format: Line and paragraph breaks are automatic, E-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Back to top

Retrieval statistics: 21 queries taking a total of 0.104 seconds • Please report low bandwidth using the feedback form
Original styles created by Ian Main (all acknowledgements) • PHP scripts and styles later modified by Roy Schestowitz • Help yourself to a GPL'd copy
|— Proudly powered by W o r d P r e s s — based on a heavily-hacked version 1.2.1 (Mingus) installation —|