Storage

File System

The global filesystem on Opuntia is NFS.  This is the file system for all home areas, /home, and a larger work area, /project.

Disk Quota and Backup

Each user will have a home directory under the /home filesystem. Each user will have a /home/username directory created when their account is opened. This directory is available on the login node as well as all the other nodes in the cluster. Users will be given a 10GB quota on the /home filesystem. This space is for relatively long-term storage of programs and other files used in preparing programs and scripts and submitting/managing batch jobs. It will be backed up weekly. We strongly urge you to keep a separate backup of your home directory.
The default /project quota for a group is 1TB. Requests for slightly more space can be made via a support ticket. Much larger needs will be handled through consultation with the requesting group.
Note: I/O intensive jobs executing on the compute nodes should refrain from using the /home file system, but should instead use the file systems such as the compute node’s local /tmp. Any compute node job doing extensive I/O to the /home file system will be subject to being killed by the sys-admin team on behalf of the user community.

The home and project areas are backed up daily incrementally, that is all files modified since last full backup are copied. A full backup is carried out monthly.

Transferring Data

Basic Tools

See this  section in the Getting Started documents.

Compressing/uncompressing files and directories

The available UNIX-commands for compression and uncompression are:

• • gzip/gunzip – for description see ‘man gzip’

• • bzip2/bunzip2 – for a description see ‘man bzip2’

To compress a file, follow the example:

$ ls -l datafile 
-rw-r--r-- 1 mad xno 1536000 2010-07-28 14:17 datafile 
$ gzip datafile 
$ ls -l datafile.gz 
-rw-r--r-- 1 mad xno 307447 2010-07-28 14:17 datafile.gz

To uncompress a gzipped file, follow the example:

$ ls -l datafile.gz 
-rw-r--r-- 1 mad xno 307447 2010-07-28 14:17 datafile.gz 
$ gunzip datafile.gz 
$ ls -l datafile 
-rw-r--r-- 1 mad xno 1536000 2010-07-28 14:17 datafile

To compress a directory, the compression tool needs to be used in combination with a archiving tool, like tar:

$ ls -ld project 
drwx------ 15 mad xno 32768 2009-12-09 18:25 project 
$ tar czf project.tar.gz project/ 
$ ls -l project.tar.gz 
-rw------- 1 mad xno 304827 2010-08-09 11:07 project.tar.gz

To unpack a archived directory:

$ tar xzf project.tar.gz 
$ ls -ld project 
drwx------ 15 mad xno 32768 2012-12-09 18:25 project

Scientific Data Libraries

See the NetCDF page.

See the HDF5 page.