Provided by: libmemkind-dev_1.13.0-1_amd64 bug

NAME

       libmemtier.so - interposer library which enables the memkind memory tiering

SYNOPSIS

       In order to use the memkind memory tiering, pass the following environment variables along
       with the command:

              LD_PRELOAD=libmemtier.so MEMKIND_MEM_TIERS="..."  command {arguments ...}

DESCRIPTION

       This library enables  the  memkind  memory  tiering  mechanism.  With  this  funcionality,
       allocations  will  be  split  between different types of memory automatically. The library
       allows making allocations with the usage of  multiple  kinds  keeping  a  specified  ratio
       between  them.  This  ratio  determines  how  much of the total allocated memory should be
       allocated with each kind. Both LD_PRELOAD and MEMKIND_MEM_TIERS environment variables  are
       mandatory for enabling the memkind memory tiering.

ENVIRONMENT

       MEMKIND_MEM_TIERS
              A  semicolon-separated  list  of tier configurations ended with a policy parameter.
              Each of the tier configurations is a comma-separated list of tier parameters in the
              format:  param_name:value,(...)   For the list of available tier parameters, please
              see the TIER PARAMETERS section below.

       MEMKIND_MEM_THRESHOLDS
              Semicolon-separated list of threshold configurations. Each  configuration  consists
              of   a  comma-separated  list  of  threshold  parameters  in  the  same  format  as
              MEMKIND_MEM_TIERS , where the param_name  is  one  of  the  INIT_VAL,  MIN_VAL  and
              MAX_VAL.   Each  parameter  can  be  set  only once. When any of the parameters are
              missing, they will be set to  a  default  value.   If  threshold  configuration  is
              missing,  threshold  with  default  configuration  will  be  used.  If  any  of the
              parameters is set more than  once,  the  application  will  be  aborted.  Order  of
              provided  configurations  is  important  -  configuration  of Nth threshold defines
              threshold between Nth and (N+1)th tier. Note that for N+1 tiers there should be  at
              most N thresholds defined.  Set this variable only when DYNAMIC_THRESHOLD policy is
              used. For more details about param_name see the THRESHOLD PARAMETERS section below.
              See  also  EXAMPLES  section  for  the  usage of MEMKIND_MEM_THRESHOLDS environment
              variable.

TIER PARAMETERS

       KIND   (required) - kind of memory used in memory tier. Allowed  kind  names  are  'DRAM',
              'KMEM_DAX'  and  'FS_DAX'.   There can be multiple different kind names provided in
              the MEMKIND_MEM_TIERS environment  variable  -  one  in  each  tier  configuration.
              Additionally,  in  this variable more than one 'FS_DAX' kind name is allowed.  More
              information about available kinds can be found in the memkind(3) manual, where  the
              'DRAM'  kind  name allocates memory with the usage of the MEMKIND_DEFAULT kind, the
              'KMEM_DAX' with the MEMKIND_DAX_KMEM kind and the 'FS_DAX' with a file-backed  kind
              of memory created with the memkind_create_pmem() function.

       PATH   (required only for the FS_DAX kind) - the path to the location where pmem file will
              be created. The path has to exist. For other kinds, if set, will cause an error.

       PMEM_SIZE_LIMIT
              (optional, only for the FS_DAX kind) - if set, it limits the size of pmem file  and
              the  maximum size of total allocations from persistent memory. By default, no limit
              is introduced. Pass this option only with FS_DAX kind. For  other  kinds,  if  set,
              will  cause  an error.  The accepted formats are: 1, 1K, 1M, 1G. See the memkind(3)
              manual for information about limitations to this value which are the  same  as  for
              the max_size value of the memkind_create_pmem() function.

       RATIO  (required) - the part of the ratio tied to the given kind. It's an unsigned type in
              a range from 1 to UINT_MAX.  See EXAMPLES section.

       POLICY (required, only one in the whole environment variable) - determines  the  algorithm
              used to distribute allocations between provided memory kinds. This parameter has to
              be the last parameter in MEMKIND_MEM_TIERS  configuration  string.  Currently  only
              STATIC_RATIO and DYNAMIC_THRESHOLD policies are valid. See the POLICIES section.

       NOTE:  The  application  will fail when provided environment variable string is not in the
       correct format.

POLICIES

       STATIC_RATIO
              All allocations are made in such a way that  the  ratio  between  memory  tiers  is
              constant. No threshold is used.

       DYNAMIC_THRESHOLD
              The  ratio  between  memory  tiers is kept with a help of a threshold between kinds
              which value changes in time.  Minimum two  tiers  are  required  for  this  policy,
              otherwise  the  application will be aborted. The threshold value can change in time
              to keep the desired ratio between tiers, but it will not be lesser than MIN_VAL and
              it  will not be greater than MAX_VAL.  For every allocation, if its size is greater
              than or equal to INIT_VAL, it will come from the (N+1)th tier.

              Default values for a threshold between first two  tiers  in  the  MEMKIND_MEM_TIERS
              environment variable are:

              INIT_VAL = 1024, MIN_VAL = 513, MAX_VAL = 1536.

              If  there  are  more  tiers  defined,  each  next undefined threshold will have all
              parameters increased by 1024, so the next undefined threshold between the next  two
              tiers will have:

              INIT_VAL = 2048, MIN_VAL = 1537, MAX_VAL = 2560.

THRESHOLD PARAMETERS

       INIT_VAL
              (optional)  -  the  initial  value  of  the  threshold between two adjacent Nth and
              (N+1)th tiers. It must be greater than or equal to MIN_VAL and less than  or  equal
              to MAX_VAL.

       MIN_VAL
              (optional) - the minimum value of the threshold.

       MAX_VAL
              (optional) - the maximum value of the threshold.

       NOTE: Because setting the above parameters is optional, they will be set to default values
       in case they are not defined.

DRAM FALLBACK POLICY

       With the usage of both DRAM and KMEM_DAX tiers, if there is not enough memory  to  satisfy
       the DRAM tier memory allocation request, the allocation will fall back to PMEM memory.

       With the usage of both DRAM and FS_DAX tiers, if there is not enough memory to satisfy the
       DRAM tier memory allocation request, the allocation will fail.

       If there is not enough memory to satisfy the FS_DAX or  KMEM_DAX  tier  memory  allocation
       request, the allocation will fail.

EXAMPLES

       The  following example will run ls with the memkind memory tiering library. Make sure that
       paths to both libmemtier.so and libmemkind.so are included in LD_LIBRARY_PATH.  During the
       application  run, 20% of the allocated memory will come from PMEM memory and 80% will come
       from DRAM ( FS_DAX:DRAM ratio is 1:4):

              LD_PRELOAD=libmemtier.so
              MEMKIND_MEM_TIERS="KIND:FS_DAX,PATH:/mnt/pmem0,PMEM_SIZE_LIMIT:10G,RATIO:1;KIND:DRAM,RATIO:4;POLICY:STATIC_RATIO"
              /bin/ls -l

       The example value of MEMKIND_MEM_TIERS environment variable  where  all  allocations  will
       come  from PMEM memory with filesystem created with the path /mnt/pmem0 (PMEM file size is
       limited only by the specified filesystem):

              LD_PRELOAD=libmemtier.so
              MEMKIND_MEM_TIERS="KIND:FS_DAX,PATH:/mnt/pmem0,RATIO:1;POLICY:STATIC_RATIO"

       The example value of MEMKIND_MEM_THRESHOLDS environment variable. With INIT_VAL=64, on the
       application start all allocations lower than 64 bytes threshold will come  from  DRAM  and
       equal  to  or greater than this value will come from PMEM memory NUMA nodes. The threshold
       value changes  during  the  runtime  in  order  to  maintain  the  ratio.   MIN_VAL=1  and
       MAX_VAL=10000  set  the  lower  and  upper  limits  of  the threshold value. Note that the
       DYNAMIC_THRESHOLD policy has to be set in MEMKIND_MEM_TIERS environment variable:

              LD_PRELOAD=libmemtier.so
              MEMKIND_MEM_TIERS="KIND:DRAM,RATIO:1;KIND:KMEM_DAX,RATIO:4;POLICY:DYNAMIC_THRESHOLD"
              MEMKIND_MEM_THRESHOLDS="INIT_VAL=64,MIN_VAL=1,MAX_VAL=10000"

NOTES

       libmemtier works for applications that do not statically link a malloc implementation.

COPYRIGHT

       Copyright (C) 2021 Intel Corporation. All rights reserved.

SEE ALSO

       memkind(3), malloc(3)