For the purposes of the cluster we are talking about RAM memory. If you want to know more about the other types of memory please see this page.
Memory usage and swap space
As well as the physical memory, each computer has a certain amount of virtual memory, or swap space available. Swap space is an area of the hard disk drive which the Operating System uses as additional memory, using a small amount of swap space in normal circumstances. When a job requires more memory than is physically available, it will start using swap space. Because swap space is on the hard disk drive it is significantly slower than real memory, and generally if a job starts swapping, it will slow to the point it will no longer be making any progress with the task and it is best to terminate the job before the node becomes unstable.
The following graph shows a snapshot of the memory usage of a node. It is idle to start with. A job started and very quickly used all teh available memory, and began swapping (this means it becomes less efficient). There is a short period where the memory use drops - this may be a temporary change in the memory requirement of the job, or it may be the job ending, with a second job with a similar memory profile starting soon after.
How much memory is available ?
Some of the memory will be used by the Operating System for generic system related tasks.
- Compute nodes have 64, 96 or 128Gb
- GPU nodes have 384Gb
- High memory nodes have 768Gb
The memory on each node is a shared resource - the memory will be shared between all running jobs and the operating system. Standard conmpute nodes have a notional 4Gb of memory per cpu slot. Memory requirements differ between jobs, depending on what they are doing and how much data they are handling at any one time. Memory use can also vary throughout the life of a job. For this reason we ask you to specify the amount of memory your job requires, to avoid your job taking more than expected and affecting other user's jobs running on the same node.
We ask you to be as specific as possible in your memory reuqests when submitting a job. If you ask for an exclusive session you will have access to all the memory on the nodePlease work through the how much memory and requesting memory sections.