Sequential Job Scripts

The primary way to run a job on the cluster is to submit it as a batch job (no interaction with it once the job is submitted) to the relevent queue, using a submit script.  For clarity, and to help us with any support you might require, submit scripts should end in a .sub extension.

Batch Jobs can be run against any valid queue as defined in the batch script itself, and are submitted to the cluster using a cluster command called bsub, so the cluster batch job scripts are known as submit scripts.  
A batch submit script defines everything about your job, including what, how and where you want it to run. Examples of what might be configured in a submit script include;
    • Which queue you want to use
    • Whether you want to share a node, or get exclusive access to a node in the queue
    • The name of the job
    • Where you want output to go (output and error streams etc)
    • How much RAM the job will need (esp if different from the default 4Gb per job)
    • At what amount of RAM usage the job can be automatically stopped by the scheduler (to protect the cluster if something goes wrong)
    • Which modules you need to load
    • Which applications you need to run and the commands you want to run against them

Slurm job scripts can be configured to request particular cluster resource, i.e. the queue to run the job on or the name of the job. After you have added your slurm directives to your job submission script, the required modules must be loaded and finally the command is executed. 

 

slurm options

By default both standard output and standard error are directed to the same file.

slurm option description
#SBATCH --job-name=test_job Set job name
#SBATCH -p compute set queue to use
#SBATCH -o test-%j.out set output file
#SBATCH -e test-%j.err  set error output file
#SBATCH --exclusive set exclusive use of node
#SBATCH --mail-type=ALL set when to send mail events  -
options NONE, BEGIN, END, FAIL, ALL
#SBATCH --mail-user=<username>@uea.ac.uk Where to send mail
#SBATCH --mem 2G set memory limit
Different units can be specified using the suffix [K|M|G|T]
????

Use this setting to request that a certain amount of RAM be reserved (in MB e.g. 6000=6GB)* on the node that the job is submitted to.

#SBATCH --array=1-10 set an array of 10 jobs

 

Submitting jobs

 

sbatch JobScriptName.sub

Example job submit scripts

An example of a Matlab job script is shown below

#!/bin/bash
#SBATCH --mail-type=END,FAIL         #Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=<username>@uea.ac.uk     # Where to send mail
#SBATCH -p compute                              # Submit to compute queue
#SBATCH --job-name=matlab-test_job     # Set job name
#SBATCH -o matlab-test-%j.out             # Write job output to MatJob-(job_number).out
#SBATCH -e matlab-test-%j.err             # Write job error to MatJob-(job_number).err

. /etc/profile
module add matlab/2018a
matlab -nodisplay -nojvm -nodesktop -nosplash -r my_matlab_m_file

An example of a Stata15 multicore job script is shown below

#!/bin/bash
#SBATCH --mail-type=ALL           #Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=<username>@uea.ac.uk    # Where to send mail
#SBATCH -p compute                  #Which queue to use
#SBATCH --exclusive                      # set exclusive use of node
#SBATCH --job-name=stata-test_job #Job name
#SBATCH -o stata-test-%j.out       #Standard output log
#SBATCH -e stata-test-%j.err       #Standard error log
. /etc/profile
module add stata/15
stata-mp -b do stata15_example.dostata

An example of an R job is shown below

#!/bin/bash
#SBATCH --mail-type=ALL                    #Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=<username>@uea.ac.uk           # Where to send mail
#SBATCH -p compute                            #Which queue to use
#SBATCH --job-name=R-test_job        #Job name
#SBATCH -o R-test-%J.out                    #Standard output log  
#SBATCH -e R-test-%J.err                     #Standard error log
. /etc/profile
module add R/3.6.1/container
R CMD BATCH TestRFile.R