Access to the HPC system is by Secure Shell from a computer connected to the campus network, full details are provided once your application has been accepted.
Introduction
The HPC system has 28 processors, two of these are reserved for the system, leaving 26 available for dedicated processing.
CPU time on the 26 processors is managed via a queuing system. The processing queue is split into 26 "slots" with each slot representing one of the computational CPUs.
Submitting Jobs
The qsub command submits tasks to a processing queue. You must use a wrapper script to run your job and then submit the wrapper script to the scheduler.
Jobs will be held in a queue until a processing queue with enough slots is available to process the job.
Example: Submitting a task
prompt$ qsub myjob.sh
Your job 25 ("myjob.sh") has been submitted.
prompt$
In the above example, the job has been submitted to the scheduler and given a job id of 25.
Any output from your job will be saved in your home directory with names like myjob.sh.o25 and myjob.sh.e25 representing STDOUT and STDERR respectively. The "25" suffix is the job id.
By default, tasks will be provided with one CPU slot per job. To run SMP jobs on more than one processor you must request a parallel environment or PE.
Example: Requesting a parallel environment for a job
prompt$ qsub -pe smp 4 mysmpjob.sh
The example above requests a parallel environment of four CPUs which will use four slots on the processing queue. If there are not enough slots available to process the request it will be queued until they become available.
Monitoring Jobs & Queues
The qstat command shows the progress of your jobs and allows you to see the state of the available queues.
Example: Viewing queue usage
prompt$ qstat -f
queuename qtype used/tot. load_avg arch states
-----------------------------------------------------------
smp.q@hpc BIP 5/26 1.03 lx24-ia64
12 0.56000 job1.sh nuser t 01/19/2005 10:31:27 1
13 0.61000 job2.sh nuser t 01/19/2005 11:44:42 4
In the example above, the smp.q queue has two jobs running on it, using a total of 5 slots. Job id 12 is running on one CPU and job id 13 is running with four CPUs.
Removing Jobs From The Queue
The qdel command is to remove jobs you no longer wish to be processed. Example:
prompt$ qdel 25 nuser has deleted job 25
Queuing System GUI
If you have X11 forwarding enabled in your SSH session you may prefer to use the graphical interface to the queuing system. To run simply type qmon at the shell prompt.
Wrapper scripts are used to request resources from the batch scheduler and set options that your job needs to run properly. They are essentially shell scripts which may contain some extra options.
Example: Basic wrapper script
# Wrapper script to start my processing job # # Lines prefixed with '#' are ignored by the system # $HOME/my-program-file > $HOME/program-log
The above shows a basic wrapper that runs the program 'my-program-file' from your home directory and sends any output to a file called 'program-log', also in your home directory.
Script lines that begin with #$ set options you would otherwise put on the qsub command line. For example, if your qsub command looked like this:
prompt$ qsub -pe smp 12 mysmpjob.sh
Then you can simplify by adding the options to your wrapper script:
# Wrapper script to start my processing job # # Lines prefixed with '#' are ignored by the system # #$ -pe smp 12 # $HOME/my-program-file > $HOME/program-log
Now you can submit your task just by typing:
prompt$ qsub mysmpjob.sh
You can find all of the available options in the qsub man page.
More general information on using Unix systems can be found on the ISS Unix Team's Unix help pages:
Detailed information and manuals for the job queuing system (Sun Grid Engine) is available at the following URL:
Return to HPC index.