What is Conda?

Conda is an open-source package management system and environment management system. Conda quickly installs, runs, and updates packages and their dependencies. Conda easily creates, saves, loads, and switches between environments.  It was created for Python programs but it can package and distribute software for any language.  In our contect it is usually used for python, but could also be used for R, java, fortran and many others.  

Using Conda on ADA

Conda on ADA is based on Anaconda.

There are a lot of commands within conda - please see the user guide and download the cheat sheet.
A few basic commands are given below

  • To enable conda

module add python/anaconda/2019.10/3.7

  • To verify the conda environment

conda --version

  • To find the python version

python --version

  • To list installed software
  • conda list

Run conda interactively

[s154@login02 ~]$ interactive
srun: job 222161 queued and waiting for resources
srun: job 222161 has been allocated resources
[s154@c0002 ~]$ module add python/anaconda/2019.10/3.7

Run conda in a batch job

#!/bin/bash
#SBATCH --mail-type=END,FAIL    # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=<username>@uea.ac.uk     # Where to send mail
#SBATCH -p compute-24-96             # Select compute-24-96 queue
#SBATCH -t 36:00:00             # Set time limit to 36 hours
#SBATCH --job-name=conda-test_job      # Set job name
#SBATCH -o conda-test-%j.out               # Write job output to conda-(job_number).out
#SBATCH -e conda-test-%j.err               # Write job error to conda-(job_number).err
#set up environment
module add python/anaconda/2019.10/3.7
#run the application
conda --version
.......

Conda environments

A conda environment is a directory that contains a specific collection of conda packages that you have installed. For example, you may have one environment with NumPy 1.7 and its dependencies, and another environment with NumPy 1.6 for legacy testing.

If you are using conda for a number of types of jobs, you can set up and environment for each work stream.  Any environments you create will be specific to your account.  If you use a different conda module (eg from previous use on hpc.uea.ac.uk, or when there are more than one anaconda module available), they will only be visible when the appropriate conda module is loaded.

By default conda uses the base environment (ie the one provided by the module add python/anaconda/2019.10/3.7 command)

  • To see which environments you have available

conda info --envs
# conda environments:
#
base                  *  /gpfs/software/ada/python/anaconda/2019.10/3.7
ncl_stable               /gpfs/software/ada/python/anaconda/2019.10/3.7/envs/ncl_stable
qiime2-2019.10           /gpfs/software/ada/python/anaconda/2019.10/3.7/envs/qiime2-2019.10

Create a conda environment

this has to be done in an interactive sessaion.  Once created it can be used in batch jobs or interactive sessions.

Note: Takes about 5-10 minutes to complete.

  • interactive
  • module add python/anaconda/2019.10/3.7
  • conda create -n yourenvname python=3.7 anaconda

Activate your environment

Either in batch jobs or interactive sessions.

conda activate yourenvname 
(in batch jobs it might be better to use source activate yourenvname)
conda init bash (this is only needed the first time you activate your environment)

Install packages in you environment

just as you would install a package in python, but spectify your environment

conda install -n yourenvname [package]

Deactivate your environment

source deavtivate

Remove an environment

conda remove -n yourenvname -all