Ubuntu Manpage: likwid-pin - pin a sequential or threaded application to dedicated processors

name
synopsis
description
options
example
author
bugs
see also

NAME

       likwid-pin - pin a sequential or threaded application to dedicated processors

SYNOPSIS

       likwid-pin [-vhqipS] [-c <core_list>] [-s <skip_mask>] [-d <delimiter>]

DESCRIPTION

likwid-pin is a command line application to pin a sequential or multithreaded applications to dedicated
processors. It can be used as replacement for taskset(1). Opposite to taskset no affinity mask but
single processors are specified. For multithreaded applications based on the pthread library the
pthread_create library call is overloaded through LD_PRELOAD and each created thread is pinned to a
dedicated processor as specified in core_list

Per default every generated thread is pinned to the core in the order of calls to pthread_create. It is
possible to skip single threads using -s commandline option.

For OpenMP implementations gcc and icc compilers are explicitly supported. Others may also work. likwid-
pin sets the environment variable OMP_NUM_THREADS for you if not already present. It will set as many
threads as present in the pin expression. Be aware that with pthreads the parent thread is always
pinned. If you create for example 4 threads with pthread_create and do not use the parent process as
worker you still have to provide num_threads+1 processor ids.

likwid-pin supports different numberings for pinning. Per default physical numbering of the cores is
used. This is the numbering also likwid-topology(1) reports. But also logical numbering inside the node
or the sockets can be used. If using with a N (e.g. -c N:0-6) the cores are logical numbered over the
whole node. Physical cores come first. If a system e.g. has 8 cores with 16 SMT threads with -c N:0-7
you get all physical cores. If you specify -c N:0-15 you get all physical cores and all SMT threads.
With S you can specify logical numberings inside sockets, again physical cores come first. You can mix
different domains separated with @. E.g. -c S0:0-3@S2:2-3 you pin thread 0-3 to logical cores 0-3 on
socket 0 and threads 4-5 on logical cores 2-3 on socket 2.

For applications where first touch policy on numa systems cannot be employed likwid-pin can be used to
turn on interleave memory placement. This can significantly speed up the performance of memory bound
multithreaded codes. All numa nodes the user pinned threads to are used for interleaving.

OPTIONS

       -v     prints version information to standard output, then exits.

       -h     prints a help message to standard output, then exits.

       -c  <processor_list> OR <thread_expression> OR <scatter policy>
              specify  a  numerical list of processors. The list may contain multiple items, separated by comma,
              and ranges. For example 0,3,9-11. You can also use logical numberings, either within a node (N), a
              socket (S<id>) or a numa domain (M<id>).  likwid-pin also supports logical pinning within a cpuset
              with a L prefix. If you ommit this option likwid-pin will pin the threads to the processors on the
              node  with  physical  cores  first.  See below for details on using a thread expression or scatter
              policy

       -s  <skip_mask>
              Specify skip mask as HEX number. For each set bit the corresponding thread is skipped.

       -S     All ccNUMA memory domains belonging to the specified threadlist will be cleaned  before  the  run.
              Can solve file buffer cache problems on Linux.

       -p     prints  the  available  thread  domains  for  logical pinning. If used in combination with -c, the
              physical processor IDs are printed to stdout.

       -i     set numa memory policy to interleave spanning all numa nodes involved in pinning

       -q     silent execution without output

       -d  <delimiter>
              set delimiter used to output the physical processor list (-p & -c)

EXAMPLE

1. For standard pthread application:

likwid-pin -c 0,2,4-6 ./myApp

The parent process is pinned to processor 0. Thread 0 to processor 2, thread 1 to processor 4, thread 2
to processor 5 and thread 3 to processor 6. If more threads are created than specified in the processor
list, these threads are pinned to processor 0 as fallback.

2. For gcc OpenMP as many ids must be specified in processor list as there are threads:

OMP_NUM_THREADS=4; likwid-pin -c 0,2,1,3 ./myApp

3. Full control over the pinning can be achieved by specifying a skip mask. For example the following
command skips the pinning of thread 1:

OMP_NUM_THREADS=4; likwid-pin -s 0x1 -c 0,2,1,3 ./myApp

4. The -c switch supports the definition of threads in a specific affinity domain like NUMA node or
cache group. The available affinity domains can be retrieved with the -p switch and no further option
on the commandline. The common affinity domains are N (whole Node), SX (socket X), CX (cache group X)
and MX (memory group X). Multiple affinity domains can be set separated by @. In order to pin 2
threads on each socket of a 2-socket system:

OMP_NUM_THREADS=4; likwid-pin -c S0:0-1@S1:0-1 ./myApp

5. Another argument definition of the -c switch allows the threads to be pinned according to an
expression like E:N:4:1:2. The syntax is E:<thread domain>:<number of threads>(:<chunk
size>:<stride>). The example pins 8 threads with 2 SMT threads per core on a SMT 4 machine:

OMP_NUM_THREADS=4; likwid-pin -c E:N:8:2:4 ./myApp

6. The last alternative for the -c switch is the automatic scattering of threads on affinity domains.
For example to scatter the threads over all memory domains in a system:

OMP_NUM_THREADS=4; likwid-pin -c M:scatter ./myApp

AUTHOR

       Written by Jan Treibig <jan.treibig@gmail.com>.

BUGS

       Report Bugs on <http://code.google.com/p/likwid/issues/list>.