Provided by: numad_0.5+20150602-5ubuntu1_amd64 bug


       numad  -  A  user-level  daemon  that provides placement advice and process management for
       efficient use of CPUs and memory on systems with NUMA topology.


       numad [-dhvV]

       numad  [-C 0|1]

       numad  [-H THP_hugepage_scan_sleep_ms]

       numad  [-i [min_interval:]max_interval]

       numad  [-K 0|1]

       numad  [-l log_level]

       numad  [-m target_memory_locality]

       numad  [-p PID]

       numad  [-r PID]

       numad  [-R reserved-CPU-list]

       numad  [-S 0|1]

       numad  [-t logical_CPU_percent]

       numad  [-u target_utilization]

       numad  [-w NCPUS[:MB]]

       numad  [-x PID]


       Numad is a system daemon that monitors NUMA topology and resource usage. It  will  attempt
       to  locate  processes  for  efficient NUMA locality and affinity, dynamically adjusting to
       changing  system  conditions.   Numad  also  provides  guidance   to   assist   management
       applications  with initial manual binding of CPU and memory resources for their processes.
       Note that numad is primarily intended for server consolidation environments,  where  there
       might  be  multiple  applications  or  multiple  virtual guests running on the same server
       system.  Numad is most likely to have a positive effect when processes can be localized in
       a  subset  of  the  system's NUMA nodes.  If the entire system is dedicated to a large in-
       memory database application, for example -- especially  if  memory  accesses  will  likely
       remain unpredictable -- numad will probably not improve performance.


       -C <0|1>
              This  option  controls whether or not numad treats inactive file cache as available
              memory. By default, numad assumes it can count inactive file cache as "free" memory
              when  considering  resources to match with processes.  Specify -C 0 if numad should
              instead consider inactive file cache as a consumed resource.

       -d     Debug output in log, sets the log level to LOG_DEBUG.  Same effect as -l 7.

       -h     Display usage help information and then exit.

       -H  <THP_scan_sleep_ms>
              Set the desired transparent hugepage scan interval in ms.  The
              /sys/kernel/mm/tranparent_hugepage/khugepaged/scan_sleep_millisecs    tunable    is
              usually set to 10000ms by the operating system.  The default is changed by numad to
              be  1000ms  since  it is helpful for the hugepage daemon to be more aggressive when
              memory moves between nodes.  Specifying (-H 0)  will  cause  numad  to  retain  the
              system  default  value.   You  can  also  make  the  hugepage  daemon  more or less
              aggressive by specifying an alternate value with this option.  For example, setting
              this  value  to 100ms (-H 100) might improve the performance of workloads which use
              many transparent hugepages.

       -i <[min_interval:]max_interval>
              Sets the time interval that  numad  waits  between  system  scans,  in  seconds  to
              <max_interval>.  Default  <max_interval> is 15 seconds, default <min_interval> is 5
              seconds.  Setting a <max_interval> of zero will cause the daemon to exit.  (This is
              the  normal  mechanism  to  terminate  the  daemon.)   A bigger <max_interval> will
              decrease numad overhead but also decrease responsiveness to  changing  loads.   The
              default numad max_interval can be changed in the numad.conf file.

       -K <0|1>
              This  option  controls  whether  numad  keeps interleaved memory spread across NUMA
              nodes, or attempts to merge interleaved memory to local NUMA nodes.  The default is
              to merge interleaved memory.  This is the appropriate setting to localize processes
              in a subset of the system's NUMA nodes.   If  you  are  running  a  large,  single-
              instance  application  that  allocates interleaved memory because the workload will
              have continuous unpredictable  memory  access  patterns  (e.g.  a  large  in-memory
              database),  you  might  get  better results by specifying -K 1 to instruct numad to
              keep interleaved memory distributed.

       -l <log_level>
              Sets the log level to <log_level>.  Reasonable choices are 5, 6, or 7.  The default
              value  is  5.  Note that CPU values are scaled by a factor of 100 internally and in
              the numad log files.  Unfortunately, you don't actually have that many CPUs.

       -m  <target_memory_locality>
              Set the desired memory locality threshold to stop  moving  process  memory.   Numad
              might  stop  retrying  to coalesce process memory when more than this percentage of
              the process's memory is already localized in the target node(s).   The  default  is
              90%.  Numad  will frequently localize more than the localization threshold percent,
              but it will not necessarily do so.  Decrease the threshold to allow numad to  leave
              more  process  memory  distributed  on  various  nodes.   Increase the threshold to
              instruct numad to try to localize more memory.  Acceptable values  are  between  50
              and  100 percent.  Note that setting the target memory locality to 100% might cause
              numad to continually retry to move memory that the kernel will  never  successfully

       -p <PID>
              Add  PID  to  explicit inclusion list of processes to consider for managing, if the
              process also uses significant resources.  Multiple -p PID options can be  specified
              at daemon start, but after daemon start, only one PID can be added to the inclusion
              list per subsequent numad invocation.  Use with -S to precisely control  the  scope
              of  processes  numad  can  manage.   Note  that  the  specified  process  will  not
              necessarily be actively managed unless it also meets numad's significance threshold
              -- which is currently 300MB and half of a CPU.

       -r <PID>
              Remove  PID  from both the explicit inclusion and the exclusion lists of processes.
              After daemon start, only one PID can be removed from the explicit process lists per
              subsequent  numad  invocation.   Use with -S and -p and -x to precisely control the
              scope of processes numad can manage.

       -R <CPU_LIST>
              Specify a list of CPUs that numad should assume are reserved for non-numad use.  No
              processes  will  be bound to the specified CPUs by numad.  This option is effective
              only when starting numad.  You cannot change reserved CPUs dynamically while  numad
              is already running.

       -S <0|1>
              This option controls whether numad scans all system processes or only the processes
              on the explicit inclusion PID list.  The default is to scan all processes.  Use  -S
              0  to scan only the explicit inclusion PID list.  Use -S 1 to again scan all system
              processes (excepting those on the explicit exclusion list).  Starting numad as
              numad -S 0 -p <PID-1> -p <PID-2> -p <PID-3>
              will limit scanning, and thus also automatic NUMA management, to only  those  three
              explicitly specified processes.

       -t  <logical_CPU_percent>
              Specify  the resource value of logical CPUs.  Hardware threads typically share most
              core resources, and so logical CPUs add only a  fraction  of  CPU  power  for  many
              workloads.   By  default  numad  considers  logical CPUs to be only 20 percent of a
              dedicated hardware core.

       -u  <target_utilization>
              Set the desired maximum consumption percentage of a node. Default is 85%.  Decrease
              the target value to maintain more available resource margin on each node.  Increase
              the target value to more exhaustively consume node resources.  If  you  have  sized
              your  workloads  to  precisely  fit  inside  a NUMA node, specifying (-u 100) might
              improve system performance by telling  numad  to  go  ahead  and  consume  all  the
              resources  in  each  node.   It  is possible to specify values up to 130 percent to
              oversubscribe CPUs in the nodes, but memory utilization is always capped  at  100%.
              Use oversubscription values very carefully.

       -v     Verbose output in log, sets the log level to LOG_INFO.  Same effect as -l 6.

       -V     Display version information and exit.

       -w <NCPUS[:MB]>
              Queries  numad  for  the best NUMA nodes to bind an entity that needs <NCPUS>.  The
              amount of memory (in MBs) is optional, but should normally  be  specified  as  well
              <:MB>  so  numad  can recommend NUMA nodes with available CPU capacity and adequate
              free memory.  This query option can be used regardless of whether numad is  running
              as  a  daemon.   (An  invocation  using  this option when numad is not running as a
              daemon, will not cause the daemon to start.) Output of this option is a string that
              contains a NUMA node list.  For example: 2-3,6.  The recommended node list could be
              saved in a shell variable (e.g., NODES) and then used as the node list parameter in
              numactl -m $NODES -N $NODES ...
              command.  See numactl(8).

       -x <PID>
              Add  PID  to  explicit  exclusion  list  of  processes  to blacklist from managing.
              Multiple -x PID options can be specified at daemon start, but after  daemon  start,
              only  one  PID  can be added to the exclusion list per subsequent numad invocation.
              Use with -S to precisely control the scope of processes numad can manage.






       Numad can be run as a system daemon and can be managed by the standard init mechanisms  of
       the host.

       If interactive (manual) control is desired, you can start the daemon manually by typing:


       Subsequent numad invocations while the daemon is running can be used to dynamically change
       most run-time options.

       You can terminate numad from running by typing:

       /usr/bin/numad -i0


       Bill Gray <>