Non-Uniform Memory Access (NUMA) is a computer memory design used in multiprocessing systems where the memory access time depends on the memory location relative to the processor. In NUMA, a processor can access its own local memory faster than non-local memory.
The following topics are discussed below:
A scheduler will assign application threads to processor cores. A given thread will execute on its assigned core for some period of time before being swapped out of the core to wait, as other threads are given the chance to execute. If another core becomes available, the scheduler may choose to migrate the thread.
Processor affinity refers to the persistence of associating a thread/process with particular processor resources. Exercising processor affinity ensures that memory allocations remain local to the thread/process. The scheduler takes care of the affinity in its scheduling decisions.
The numactl program runs processes with a specific NUMA
scheduling policy:
The program may not be installed by default.
The numactl package must be installed.
Prior to working with numactl, you can obtain an initial
overview of the NUMA architecture by using the command:
numactl --hardware
For example:
$> numactl --hardware available: 4 nodes (0-3) node 0 cpus: 0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96 100 104 108 112 116 120 124 128 132 136 140 node 0 size: 191458 MB node 0 free: 168114 MB node 1 cpus: 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 113 117 121 125 129 133 137 141 node 1 size: 193481 MB node 1 free: 182639 MB node 2 cpus: 2 6 10 14 18 22 26 30 34 38 42 46 50 54 58 62 66 70 74 78 82 86 90 94 98 102 106 110 114 118 122 126 130 134 138 142 node 2 size: 193526 MB node 2 free: 188102 MB node 3 cpus: 3 7 11 15 19 23 27 31 35 39 43 47 51 55 59 63 67 71 75 79 83 87 91 95 99 103 107 111 115 119 123 127 131 135 139 143 node 3 size: 193511 MB node 3 free: 187676 MB node distances: node 0 1 2 3 0: 10 21 21 21 1: 21 10 21 21 2: 21 21 10 21 3: 21 21 21 10
For more information on numactl, see numactl(8) - Linux man
page.
To take advantage of processor affinity, the Nucleus must be started by numactl.
For example:
numactl --all --membind=2 --cpunodebind=2 adastart 100
This starts database 100, binding the Nucleus process and memory to NUMA node 2.
Note
Once you have determined the best settings, put the command in a script and save the
script in a directory included in the PATH. The policy is then set for the
command and inherited by all its child processes.