Gaussian

Access to Gaussian requires registration. Please email hpc.admin to request access to Gaussian.

There is an issue with Gaussian where it runs multiple threads, effectively utilising all available resources on the host. For this reason, please ensure Gaussian is submitted with the BSUB -x directive set in your job script, giving you exclusive access to the allocated host.

Gaussian module file

-------------------------------------------------------------------
/gpfs/software/modules/che/gaussian/09C01:

module-whatis "Gaussian 09 C01"
set root /gpfs/software/gaussian/09-c01

setenv       g09root /gpfs/software/gaussian/09-c01
setenv       GAUSS_SCRDIR /local/
setenv       GAUSS_EXEDIR /gpfs/software/gaussian/09-c01/g09/bsd:/gpfs/softw
are/gaussian/09-c01/g09/local:/gpfs/software/gaussian/09-c01/g09/extras:/gpfs/so
ftware/gaussian/09-c01/g09
setenv       GAUSS_ARCHDIR /gpfs/software/gaussian/09-c01/g09/arch
setenv       GAUSS_LEXEDIR /gpfs/software/gaussian/09-c01/g09/linda-exe
setenv       GV_DIR /gpfs/software/gaussian/09-b01-gv/gv/
setenv       _DSM_BARRIER SHM
setenv       PGI_TERM trace,abort
setenv       G09BASIS /gpfs/software/gaussian/09-c01/g09/basis
setenv       GAUSS_BSDDIR /gpfs/software/gaussian/09-c01/g09/bsd
append-path   PATH /gpfs/software/gaussian/09-c01/g09/bsd:/gpfs/software/gaussian/09-c01/g09/local:/gpfs/software/gaussian/
09-c01/g09/extras:/gpfs/software/gaussian/09-c01/g09
prepend-path   LD_LIBRARY_PATH /gpfs/software/gaussian/09-c01/g09/bsd:/gpfs/software/gaussian/09-c01/g09/local:/gpfs/softwar
e/gaussian/09-c01/g09/extras:/gpfs/software/gaussian/09-c01/g09
append-path   LD_LIBRARY_PATH /gpfs/software/gaussian/09-b01-gv/gv/lib

-------------------------------------------------------------------

A basic job script would look like the following (this can be copied from /gpfs/grace/samplescripts/gaussian.bsub):

#!/bin/sh
#BSUB -q long-eth
#BSUB -x
#BSUB -oo Gauss.%J.out
#BSUB -eo Gauss.%J.err
#BSUB -J Gaussian09
. /etc/profile

module add gaussian/09C01

g09 gauss_opt.inp

This script will run Gaussian in an exclusive session on a compute node. This can then be submitted as follows:

bsub < gaussian.bsub

Parallel with Linda

In order to run Gaussian in parallel with Linda, the Gaussian input file needs to include details about the number of processes and which hosts they are running on, such as:

%LindaWorkers=e0021:2,e0022:2,e0023,e0024,e0025

It may be possible to control this through the use of the GAUSS_LFLAGS environment variable, however details are vague on how to do this. The following shows extracts of the job log showing the difference between two Linda jobs running across 2 nodes (2 processes on each) and across 4 nodes (1 processes on each):

%NProcShared=4
Will use up to 4 processors via shared memory.
%LindaWorkers=cn167:2, cn010:2,
SetLPE: input flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh' "
SetLPE: new flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh'
  -mp 2 -nodelist 'cn167.private.dns.zone cn010.private.dns.zone'
  -opt 'Tsnet.cn167_private_dns_zone.speedfactor: 2'
  -opt 'Tsnet.cn010_private_dns_zone.speedfactor: 2'"
Will use up to 4 processors via Linda.

%NProcShared=4
Will use up to 4 processors via shared memory.
%LindaWorkers=cn165:1, cn168:1, cn019:1, cn030:1,
SetLPE: input flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh' "
SetLPE: new flags="-vv -opt 'Tsnet.Node.lindarsharg: ssh'
-nodelist 'e0025.private.dns.zone e0028.private.dns.zone
cn019.private.dns.zone cn030.private.dns.zone'"
Will use up to 4 processors via Linda.

The SetLPE configuration "-vv -opt 'Tsnet.Node.lindarsharg: ssh' " is set via the GAUSS_LFLAGS in the job script.

To get Linda configured correctly, at least until a more efficient solution can be found requires a few steps:

Setup the job script to request the appropriate number of slots
Setup the Gaussian input script to request the same number of processes with %NProcShared= operator
Include the following in the Gaussian input script %LindaWorkers= intentionally leaving the rest of the line blank
Use the script /gpfs/grace/samplescripts/gaussian-linda.bsub (shown below) as a basis - this will work out hosts/processors and make the necessary modifications to the Gaussian input script, before the job submits (making a backup of the Gaussian input file for reference)

#!/bin/sh
#BSUB -n 4
#BSUB -R 'span[ptile=2]'
#BSUB -oo Gauss.%J.out
#BSUB -eo Gauss.%J.err
#BSUB -J Gaussian09
. /etc/profile

module add gaussian/09C01

LINDA_HOSTS=`echo $LSB_MCPU_HOSTS | sed 's/$cn[0-9]*$ $[0-9]*$/\1:\2,/g'`
GAUSS_LFLAGS="-vv -opt 'Tsnet.Node.lindarsharg: ssh'"
export GAUSS_LFLAGS
sed -i.bak "s/%LindaWorkers=/%LindaWorkers=${LINDA_HOSTS}/g" gauss_opt-cc.inp
g09 gauss_opt-cc.inp

The important lines for Linda are those that start:

LINDA_HOSTS - retrieves information from the scheduler regarding hosts and processors and converts to the format required for Linda configuration
sed - completes the %LindaWorkers= in the Gaussian input file with the information identified above. In this case the input file is gauss_opt-cc.inp, and a backup copy of the original is made to gauss_opt-cc.inp.bak

Using Gaussview

You will need to set the X display to your local machine. This is dependent on your PC - see Connecting to hpc.uea.ac.uk

If you connect from a linux machine you need to connect to grace using ssh -Y <your userid>@hpc.uea.ac.uk.

Start an Xinteractive session, add the gaussview module, then start gaussview.

[cc@login00 ~/UserJobs/gaussian]$ Xinteractive -x
Job <49536> is submitted to queue <interactive>.
<<waiting for dispatch >>
<<starting on e0033>>
[cc@e0033 ~]$ module add gaussview
[cc@e0033 ~]$ gview

If you are using a windows pc, you will need to use Xming, log in with putty and X11 forwarding set . Start an Xinteractive session, add the gaussview module, then start gaussview.

[cc@login00 ~/UserJobs/gaussian]$ Xinteractive -x
Job <49536> is submitted to queue <interactive>.
<<waiting for dispatch>>
<<starting on e0033>>
[cc@e0033 ~]$ module add gaussview
[cc@e0033 ~]$ gview -soft