Gaussian
Access to Gaussian requires registration. Please email hpc.admin to request access to Gaussian.
There is an issue with Gaussian where it runs multiple threads, effectively utilising all available resources on the host. For this reason, please ensure Gaussian is submitted with the BSUB -x directive set in your job script, giving you exclusive access to the allocated host.
Gaussian module file
-------------------------------------------------------------------
/gpfs/software/modules/che/gaussian/16B01:
module-whatis "Gaussian 16 B01"
set root /gpfs/software/gaussian/16-b01
setenv g16root /gpfs/software/gaussian/16-b01
setenv GAUSS_SCRDIR /local/
setenv GAUSS_EXEDIR /gpfs/software/gaussian/16-b01/g16/bsd:/gpfs/softw
are/gaussian/16-b01/g16/local:/gpfs/software/gaussian/16-b01/g16/extras:/gpfs/so
ftware/gaussian/16-b01/g16
setenv GAUSS_ARCHDIR /gpfs/software/gaussian/16-b01/g16/arch
setenv GAUSS_LEXEDIR /gpfs/software/gaussian/16-b01/g16/linda-exe
setenv GV_DIR /gpfs/software/gaussian/16-b01/gv
setenv _DSM_BARRIER SHM
setenv PGI_TERM trace,abort
setenv G16BASIS /gpfs/software/gaussian/16-b01/g16/basis
setenv GAUSS_BSDDIR /gpfs/software/gaussian/16-b01/g16/bsd
append-path PATH /gpfs/software/gaussian/16-b01/g16/bsd:/gpfs/software/gaus
sian/16-b01/g16/local:/gpfs/software/gaussian/16-b01/g16/extras:/gpfs/software/g
aussian/16-b01/g16:/gpfs/software/gaussian/16-b01/gv
prepend-path LD_LIBRARY_PATH /gpfs/software/gaussian/16-b01/g16/bsd:/gpfs/so
ftware/gaussian/16-b01/g16/local:/gpfs/software/gaussian/16-b01/g16/extras:/gpfs
/software/gaussian/16-b01/g16
append-path LD_LIBRARY_PATH /gpfs/software/gaussian/16-b01/gv/lib
A basic job script would look like the following (this can be copied from /gpfs/grace/samplescripts/gaussian.bsub):
#!/bin/sh
#BSUB -q long-eth
#BSUB -x
#BSUB -oo Gauss.%J.out
#BSUB -eo Gauss.%J.err
#BSUB -J Gaussian16
. /etc/profile
module add gaussian/16B01
g09 gauss_opt.inp
This script will run Gaussian in an exclusive session on a compute node. This can then be submitted as follows:
bsub < gaussian.bsub
Parallel with Linda
In order to run Gaussian in parallel with Linda, the Gaussian input file needs to include details about the number of processes and which hosts they are running on, such as:
%LindaWorkers=e0021:2,e0022:2,e0023,e0024,e0025
It may be possible to control this through the use of the GAUSS_LFLAGS environment variable, however details are vague on how to do this. The following shows extracts of the job log showing the difference between two Linda jobs running across 2 nodes (2 processes on each) and across 4 nodes (1 processes on each):
%NProcShared=4
Will use up to 4 processors via shared memory.
%LindaWorkers=cn167:2, cn010:2,
SetLPE: input flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh' "
SetLPE: new flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh'
-mp 2 -nodelist 'cn167.private.dns.zone cn010.private.dns.zone'
-opt 'Tsnet.cn167_private_dns_zone.speedfactor: 2'
-opt 'Tsnet.cn010_private_dns_zone.speedfactor: 2'"
Will use up to 4 processors via Linda.
%NProcShared=4
Will use up to 4 processors via shared memory.
%LindaWorkers=cn165:1, cn168:1, cn019:1, cn030:1,
SetLPE: input flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh' "
SetLPE: new flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh'
-nodelist 'e0025.private.dns.zone e0028.private.dns.zone
cn019.private.dns.zone cn030.private.dns.zone'"
Will use up to 4 processors via Linda.
The SetLPE configuration "-vv -opt 'Tsnet.Node.lindarsharg: ssh' " is set via the GAUSS_LFLAGS in the job script.
To get Linda configured correctly, at least until a more efficient solution can be found requires a few steps:
- Setup the job script to request the appropriate number of slots
- Setup the Gaussian input script to request the same number of processes with %NProcShared= operator
- Include the following in the Gaussian input script %LindaWorkers= intentionally leaving the rest of the line blank
- Use the script /gpfs/grace/samplescripts/gaussian-linda.bsub (shown below) as a basis - this will work out hosts/processors and make the necessary modifications to the Gaussian input script, before the job submits (making a backup of the Gaussian input file for reference)
#!/bin/sh
#BSUB -n 4
#BSUB -R 'span[ptile=2]'
#BSUB -oo Gauss.%J.out
#BSUB -eo Gauss.%J.err
#BSUB -J Gaussian09
. /etc/profile
module add gaussian/16B01
LINDA_HOSTS=`echo $LSB_MCPU_HOSTS | sed 's/\(cn[0-9]*\) \([0-9]*\)/\1:\2,/g'`
GAUSS_LFLAGS="-vv -opt 'Tsnet.Node.lindarsharg: ssh'"
export GAUSS_LFLAGS
sed -i.bak "s/%LindaWorkers=/%LindaWorkers=${LINDA_HOSTS}/g" gauss_opt-cc.inp
g09 gauss_opt-cc.inp
The important lines for Linda are those that start:
- LINDA_HOSTS - retrieves information from the scheduler regarding hosts and processors and converts to the format required for Linda configuration
- sed - completes the %LindaWorkers= in the Gaussian input file with the information identified above. In this case the input file is gauss_opt-cc.inp, and a backup copy of the original is made to gauss_opt-cc.inp.bak
Using Gaussview
You will need to set the X display to your local machine. This is dependent on your PC - see Connecting to hpc.uea.ac.uk
If you connect from a linux machine you need to connect to HPC using ssh -Y <your userid>@hpc.uea.ac.uk.
Start an Xinteractive session, add the gaussview module, then start gaussview.
[cc@login00 ~/UserJobs/gaussian]$ Xinteractive -x
Job <49536> is submitted to queue <interactive>.
<<waiting for dispatch >>
<<starting on e0033>>
[cc@e0033 ~]$ module add gaussview
[cc@e0033 ~]$ gview
If you are using a windows pc, you will need to use Xming, log in with putty and X11 forwarding set . Start an Xinteractive session, add the gaussview module, then start gaussview.
[cc@login00 ~/UserJobs/gaussian]$ Xinteractive -x
Job <49536> is submitted to queue <interactive>.
<<waiting for dispatch>>
<<starting on e0033>>
[cc@e0033 ~]$ module add gaussview
[cc@e0033 ~]$ gview -soft