NUMA Architecture (Linux only)

Non-Uniform Memory Access (NUMA) is a computer memory design used in multiprocessing systems where the memory access time depends on the memory location relative to the processor. In NUMA, a processor can access its own local memory faster than non-local memory.

The following topics are discussed below:


Processor Affinity

A scheduler will assign application threads to processor cores. A given thread will execute on its assigned core for some period of time before being swapped out of the core to wait, as other threads are given the chance to execute. If another core becomes available, the scheduler may choose to migrate the thread.

Processor affinity refers to the persistence of associating a thread/process with particular processor resources. Exercising processor affinity ensures that memory allocations remain local to the thread/process. The scheduler takes care of the affinity in its scheduling decisions.

Controlling NUMA Policy with numactl

The numactl program runs processes with a specific NUMA scheduling policy:

  • The program may not be installed by default.

  • The numactl package must be installed.

Prior to working with numactl, you can obtain an initial overview of the NUMA architecture by using the command:

numactl --hardware

For example:

$> numactl --hardware
available: 4 nodes (0-3)
node 0 cpus: 0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96 100 104 108 112 116 120 124 128 132 136 140
node 0 size: 191458 MB
node 0 free: 168114 MB
node 1 cpus: 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 113 117 121 125 129 133 137 141
node 1 size: 193481 MB
node 1 free: 182639 MB
node 2 cpus: 2 6 10 14 18 22 26 30 34 38 42 46 50 54 58 62 66 70 74 78 82 86 90 94 98 102 106 110 114 118 122 126 130 134 138 142
node 2 size: 193526 MB
node 2 free: 188102 MB
node 3 cpus: 3 7 11 15 19 23 27 31 35 39 43 47 51 55 59 63 67 71 75 79 83 87 91 95 99 103 107 111 115 119 123 127 131 135 139 143
node 3 size: 193511 MB
node 3 free: 187676 MB
node distances:
node     0    1    2    3 
  0:   10   21   21   21 
  1:   21   10   21   21 
  2:   21   21   10   21 
  3:   21   21   21   10

For more information on numactl, see numactl(8) - Linux man page.

Starting Adabas with numactl

To take advantage of processor affinity, the Nucleus must be started by numactl.

For example:

numactl --all --membind=2 --cpunodebind=2 adastart 100

This starts database 100, binding the Nucleus process and memory to NUMA node 2.

Note
Once you have determined the best settings, put the command in a script and save the script in a directory included in the PATH. The policy is then set for the command and inherited by all its child processes.