How to log in

The only way to connect to Opuntia is by secure shell (ssh), e.g. from a Linux/UNIX system. The full hostname for the Opuntia cluster login node is opuntia.cacds.uh.edu:

ssh -l your_username  opuntia.rcdc.uh.edu

Windows users will need an SSH client installed on their machine, see e.g.  PuTTY or XShell.

X11 Forwarding

X11 forwarding is necessary to display editor windows (gvim, emacs, nedit, etc.) or similar on your desktop. To enable X11 forwarding, log in with the ssh -X or -Y options enabled

ssh -XY -l your_username  opuntia.rcdc.uh.edu

Windows users need an X server to handle the local display in addition to the ssh program, see  this intro (from the University of Indiana) for PuTTY users.

Transferring Data

Basic Tools

SCP (Secure CoPy): scp uses ssh for data transfer, and uses the same authentication and provides the same security as ssh. For example, copying from a local system to Opuntia:

scp myfile  your_username@opuntia.rcdc.uh.edu:

SFTP (Secure File Transfer Protocol): sftp is a file transfer program, similar to ftp, which performs all operations over an encrypted ssh transport. Example,  put file from local system to Opuntia:

sftp uusername@Opuntia.rcdc.uh.edu 
Password: 
Connected to Opuntia.rcdc.uh.edu 

sftp> put myfile
	

For Windows users  WinSCP is a free graphical SCP and SFTP client.

Software environment

Text editors

Opuntia has multiple editors installed including vi, emacs and nano.

Modules

Modules are a tool for users to manage the Unix environment in Opuntia. It is designed to simplify login scripts. A single user command,

module add module_name

can be invoked to source the appropriate environment information within the user’s current shell. Invoking the command,

module avail

will list the available packages on Opuntia

module rm module_name

Will remove the module from your environment

Running your first program

The Concept

MPI programs are executed as one or more processes; one process is typically assigned to one physical processor core. All the processes run the exact same program, but by receiving different input they can be made to do different tasks. The most common way to differ the processes is by their rank. Together with the total number of processes, referred to as size, they form the basic method of dividing the tasks between the processes. Getting the rank of a process and the total number of processes is therefore the goal of this example. Furthermore, all MPI related instructions must be issued between MPI_Init() and MPI_Finalize(). Regular C instructions that is to be run locally for each process, e.g. some preprocessing that is equal for all processes, can be run outside the MPI context.

The Code

Below is a simple program that, when executed, will make each process print their name and rank as well as the total number of processes.

/*  Basic MPI Example - Hello World  */  
#include <stdio.h> /* printf and BUFSIZ defined there */ 
#include <stdlib.h> /* exit defined there */ 
#include "mpi.h" /* all MPI-2 functions defined there */   

int main(argc, argv) 
int argc; 
char *argv[]; 
{ 
int rank, size, length; 
char name[BUFSIZ];   

MPI_Init(&argc, &argv); 
MPI_Comm_rank(MPI_COMM_WORLD, &rank); 
MPI_Comm_size(MPI_COMM_WORLD, &size); 
MPI_Get_processor_name(name, &length);   

printf("%s: hello world from process %d of %dn", name, rank, size);   

MPI_Finalize();   

exit(0); 
}

• • MPI_Init(); Is responsible for spawning processes and setting up the communication between them. The default communicator (collection of processes) MPI_COMM_WORLD is created.

• • MPI_Finalize(); End the MPI program.

• • MPI_Comm_rank( MPI_COMM_WORLD, &rank ); Returns the rank of the process within the communicator. The rank is used to divide tasks among the processes. The process with rank 0 might get some special task, while the rank of each process might correspond to distinct columns in a matrix, effectively partitioning the matrix between the processes.

• • MPI_Comm_size( MPI_COMM_WORLD, &size ); Returns the total number of processes within the communicator. This can be useful to e.g. know how many columns of a matrix each process will be assigned.

• • MPI_Get_processor_name( name, &length ); Is more of a curiosity than necessary in most programs; it can assure us that our MPI program is indeed running on more than one computer/node.

Compile & Run

Save the code in a file named helloworld.c. Load the Intel compiler and Intel MPI module files:

module load intelmpi
	

mpicc -o helloworld helloworld.c
	

Make a batch job. Add the following in a file named job.sh

#!/bin/bash 
#SBATCH -J my_mpi_job 
#SBATCH -o my_mpi_job.o%j 
#SBATCH -t 00:01:00 
#SBATCH -N 2 -n 10   

module load intelmpi   
mpirun ./helloworld

sbatch job.sh
Submitted batch job 906

Note that the command sbatch returns the job ID. See the queue status. Note that the example runs fast. It can be finished before the status command returns a job identifier. The job identifier is used to name the output from the job together with the name of the job. The job name is given with -N option in the job.sh-script. In this example it is ‘my_mpi_job’. The standard output from the processes are logged to a log file in the working directory named my_mpi_job.o. Here is the content from on batch execution of the job.sh:

			
cat my_mpi_job.o906 

			
compute-2-13.local: hello world from process 9 of 10 
compute-2-12.local: hello world from process 1 of 10 
compute-2-12.local: hello world from process 3 of 10 
compute-2-12.local: hello world from process 5 of 10 
compute-2-12.local: hello world from process 6 of 10 
compute-2-12.local: hello world from process 7 of 10 
compute-2-12.local: hello world from process 8 of 10 
compute-2-12.local: hello world from process 0 of 10 
compute-2-12.local: hello world from process 2 of 10 
compute-2-12.local: hello world from process 4 of 10
		

More Examples

To open an interactive session on a compute node

 srun  --pty  /bin/bash   -l

Same as above, but requesting 1 hour of wall time  and  X11 forwarding support

 srun  -t 1:00:00  --x11=first  --pty  /bin/bash  -l

 srun  -t  1:00:00  -n 20  -N  1 --pty  /bin/bash  -l