LSF
Platform LSF
At the heart of the cluster is the software which manages the workload. Grace uses Platform LSF which is part of the Platform HPC suite from IBM. Essentially it's a program that balances utilisation across available resource.
Basic LSF commands
| Command | Description |
|---|---|
| bjobs | Shows a list of your running jobs |
| bjobs -x | Lists your running jobs and also shows the nodes which they are running on |
| bjobs -u all | Lists all jobs running on the cluster |
| bjobs -u all -x | As above but detailing the executing node for each job |
| bjobs -l JOBID | Shows detailed information about your job (JOBID = job number) |
| bqueues | Shows a list of available queues including jobs and slots |
| bfree | Shows detailed information about the available resource (queues/nodes) |
| bkill JOBID | Kill job specified with JOBID |
| bkill 0 | Kill all of your running jobs |
| bsub | Submit a job |
BSUB commands
The BSUB commands listed below should be added to your submission scripts. If you are in doubt as to which directives to use, please email the HPC support team.
| Command | Description |
|---|---|
| #BSUB -B | Email when job starts (you don't need to supply your email) |
| #BSUB -N | Email when job finishes (you don't need to supply your email) |
| #BSUB -o output.out | Write output to output.log (use %J to include JOBID) will append to output.log if it exists |
| #BSUB -oo output.out | As above but overwriting output.out if it exists |
| #BSUB -e error.err | Write errors to error.err (use %J to include JOBID) "error-%J.err") |
| #BSUB -eo error.err | As above but overwriting error.err if it exists |
| #BSUB -x | Grants your job exclusive access to the node (Check with HPC admin prior to using) |
| #BSUB -n 16 | Request a number of slots (16 in this case) |
| #BSUB -J Jobname | Job Name |
| #BSUB -J Jobname[1-10] | Array job with 10 elements |
| #BSUB -R "rusage[mem=4000]" | Request a specific amount of memory for your job |
| #BSUB -R 'select[ivy]' | Run job on newer Sandybridge 20 core nodes only |
| #BSUB -R 'select[sandy]' | Run job on newer Sandybridge 16 core nodes only |
| #BSUB -R 'select[west]' | Run job on non Westmere 12 core nodes only |
| #BSUB -M | Sets the memory limit (MB) |
| #BSUB -u email-address | You could use this option to send an email to an external account if you so wished |
| #BSUB -q medium | Submit your job to a specific queue (medium in this example) |