Platform LSF

At the heart of the cluster is the software which manages the workload.  Grace uses Platform LSF which is part of the Platform HPC suite from IBM. Essentially it's a program that balances utilisation across available resource.

Basic LSF commands

Command Description
bjobs Shows a list of your running jobs
bjobs -x Lists your running jobs and also shows the nodes which they are running on
bjobs -u all Lists all jobs running on the cluster
bjobs -u all -x As above but detailing the executing node for each job
bjobs -l JOBID Shows detailed information about your job (JOBID = job number)
bqueues Shows a list of available queues including jobs and slots
bfree Shows detailed information about the available resource (queues/nodes)
bkill JOBID Kill job specified with JOBID
bkill 0 Kill all of your running jobs
bsub Submit a job


BSUB commands

The BSUB commands listed below should be added to your submission scripts. If you are in doubt as to which directives to use, please email the HPC support team.

Command Description
#BSUB -B Email when job starts (you don't need to supply your email)
#BSUB -N Email when job finishes (you don't need to supply your email)
#BSUB -o output.out Write output to output.log (use %J to include JOBID) will append to output.log if it exists
#BSUB -oo output.out  As above but overwriting output.out if it exists
#BSUB -e error.err Write errors to error.err (use %J to include JOBID) "error-%J.err")
#BSUB -eo error.err As above but overwriting error.err if it exists
#BSUB -x Grants your job exclusive access to the node (Check with HPC admin prior to using)
#BSUB -n 16 Request a number of slots (16 in this case)
#BSUB -J Jobname Job Name
#BSUB -J Jobname[1-10] Array job with 10 elements
#BSUB -R "rusage[mem=4000]" Request a specific amount of memory for your job
#BSUB -R 'select[ivy]' Run job on newer Sandybridge 20 core nodes only
#BSUB -R 'select[sandy]' Run job on newer Sandybridge 16 core nodes only
#BSUB -R 'select[west]' Run job on non Westmere 12 core nodes only
#BSUB -M Sets the memory limit (MB)
#BSUB -u email-address You could use this option to send an email to an external account if you so wished
#BSUB -q medium Submit your job to a specific queue (medium in this example)