Ubuntu Manpage: libmemtier.so - interposer library which enables the memkind memory tiering

Provided by: libmemkind-dev_1.14.0-3build1_amd64

NAME

       libmemtier.so - interposer library which enables the memkind memory tiering

SYNOPSIS

       In  order  to  use  the  memkind  memory  tiering with pre-built binaries, pass the following environment
       variables along with the command:

              LD_PRELOAD=libmemtier.so MEMKIND_MEM_TIERS="..."  command {arguments ...}

DESCRIPTION

       This library enables the memkind memory tiering mechanism. With this funcionality,  allocations  will  be
       split  between  different  types  of memory automatically. The library allows making allocations with the
       usage of multiple kinds keeping a specified ratio between them. This ratio determines  how  much  of  the
       total  allocated  memory  should  be  allocated  with  each  kind.  Both LD_PRELOAD and MEMKIND_MEM_TIERS
       environment variables are mandatory for enabling the memkind memory tiering.

ENVIRONMENT

MEMKIND_MEM_TIERS
A semicolon-separated list of tier configurations ended with a policy parameter. Each of the tier
configurations is a comma-separated list of tier parameters in the format: param_name:value,(...)
For the list of available tier parameters, please see the TIER PARAMETERS section below.

MEMKIND_MEM_THRESHOLDS
Semicolon-separated list of threshold configurations. Each configuration consists of a comma-
separated list of threshold parameters in the same format as MEMKIND_MEM_TIERS , where the
param_name is one of the INIT_VAL, MIN_VAL and MAX_VAL. Each parameter can be set only once. When
any of the parameters are missing, they will be set to a default value. If threshold
configuration is missing, threshold with default configuration will be used. If any of the
parameters is set more than once, the application will be aborted. Order of provided
configurations is important - configuration of Nth threshold defines threshold between Nth and
(N+1)th tier. Note that for N+1 tiers there should be at most N thresholds defined. Set this
variable only when DYNAMIC_THRESHOLD policy is used. For more details about param_name see the
THRESHOLD PARAMETERS section below. See also EXAMPLES section for the usage of
MEMKIND_MEM_THRESHOLDS environment variable.

TIER PARAMETERS

KIND (required) - kind of memory used in memory tier. Allowed kind names are 'DRAM', 'KMEM_DAX' and
'FS_DAX'. There can be multiple different kind names provided in the MEMKIND_MEM_TIERS
environment variable - one in each tier configuration. Additionally, in this variable more than
one 'FS_DAX' kind name is allowed. More information about available kinds can be found in the
memkind(3) manual, where the 'DRAM' kind name allocates memory with the usage of the
MEMKIND_DEFAULT kind, the 'KMEM_DAX' with the MEMKIND_DAX_KMEM kind and the 'FS_DAX' with a file-
backed kind of memory created with the memkind_create_pmem() function.

PATH (required only for the FS_DAX kind) - the path to the location where pmem file will be created.
The path has to exist. For other kinds, if set, will cause an error.

PMEM_SIZE_LIMIT
(optional, only for the FS_DAX kind) - if set, it limits the size of pmem file and the maximum
size of total allocations from persistent memory. By default, no limit is introduced. Pass this
option only with FS_DAX kind. For other kinds, if set, will cause an error. The accepted formats
are: 1, 1K, 1M, 1G. See the memkind(3) manual for information about limitations to this value
which are the same as for the max_size value of the memkind_create_pmem() function.

RATIO (required) - the part of the ratio tied to the given kind. It's an unsigned type in a range from 1
to UINT_MAX. See EXAMPLES section.

POLICY (required, only one in the whole environment variable) - determines the algorithm used to
distribute allocations between provided memory kinds. This parameter has to be the last parameter
in MEMKIND_MEM_TIERS configuration string. Currently only STATIC_RATIO and DYNAMIC_THRESHOLD
policies are valid. See the POLICIES section.

NOTE: The application will fail when provided environment variable string is not in the correct format.

POLICIES

       STATIC_RATIO
              All  allocations  are  made  in  such  a  way  that the ratio between memory tiers is constant. No
              threshold is used.

       DYNAMIC_THRESHOLD
              The ratio between memory tiers is kept with a help  of  a  threshold  between  kinds  which  value
              changes  in  time.  Minimum two tiers are required for this policy, otherwise the application will
              be aborted. The threshold value can change in time to keep the desired ratio between tiers, but it
              will not be lesser than MIN_VAL and it will not be greater than MAX_VAL.  For every allocation, if
              its size is greater than or equal to INIT_VAL, it will come from the (N+1)th tier.

              Default values for a threshold between  first  two  tiers  in  the  MEMKIND_MEM_TIERS  environment
              variable are:

              INIT_VAL = 1024, MIN_VAL = 513, MAX_VAL = 1536.

              If  there are more tiers defined, each next undefined threshold will have all parameters increased
              by 1024, so the next undefined threshold between the next two tiers will have:

              INIT_VAL = 2048, MIN_VAL = 1537, MAX_VAL = 2560.

THRESHOLD PARAMETERS

       INIT_VAL
              (optional) - the initial value of the threshold between two adjacent Nth  and  (N+1)th  tiers.  It
              must be greater than or equal to MIN_VAL and less than or equal to MAX_VAL.

       MIN_VAL
              (optional) - the minimum value of the threshold.

       MAX_VAL
              (optional) - the maximum value of the threshold.

       NOTE:  Because  setting the above parameters is optional, they will be set to default values in case they
       are not defined.

DRAM FALLBACK POLICY

       With the usage of both DRAM and KMEM_DAX tiers, if there is not enough memory to satisfy  the  DRAM  tier
       memory allocation request, the allocation will fall back to PMEM memory.

       With  the  usage  of  both  DRAM and FS_DAX tiers, if there is not enough memory to satisfy the DRAM tier
       memory allocation request, the allocation will fail.

       If there is not enough memory to satisfy the FS_DAX or  KMEM_DAX  tier  memory  allocation  request,  the
       allocation will fail.

EXAMPLES

       The  following  example will run ls with the memkind memory tiering library. Make sure that paths to both
       libmemtier.so and libmemkind.so are included in LD_LIBRARY_PATH.  During the application run, 20% of  the
       allocated memory will come from PMEM memory and 80% will come from DRAM ( FS_DAX:DRAM ratio is 1:4):

              LD_PRELOAD=libmemtier.so
              MEMKIND_MEM_TIERS="KIND:FS_DAX,PATH:/mnt/pmem0,PMEM_SIZE_LIMIT:10G,RATIO:1;KIND:DRAM,RATIO:4;POLICY:STATIC_RATIO"
              /bin/ls -l

       The  example  value  of  MEMKIND_MEM_TIERS environment variable where all allocations will come from PMEM
       memory with filesystem created with the path /mnt/pmem0 (PMEM file size is limited only by the  specified
       filesystem):

              LD_PRELOAD=libmemtier.so
              MEMKIND_MEM_TIERS="KIND:FS_DAX,PATH:/mnt/pmem0,RATIO:1;POLICY:STATIC_RATIO"

       The example value of MEMKIND_MEM_THRESHOLDS environment variable. With INIT_VAL=64,  on  the  application
       start all allocations lower than 64 bytes threshold will come from DRAM and equal to or greater than this
       value will come from PMEM memory NUMA nodes. The threshold value changes during the runtime in  order  to
       maintain  the  ratio.  MIN_VAL=1 and MAX_VAL=10000 set the lower and upper limits of the threshold value.
       Note that the DYNAMIC_THRESHOLD policy has to be set in MEMKIND_MEM_TIERS environment variable:

              LD_PRELOAD=libmemtier.so
              MEMKIND_MEM_TIERS="KIND:DRAM,RATIO:1;KIND:KMEM_DAX,RATIO:4;POLICY:DYNAMIC_THRESHOLD"
              MEMKIND_MEM_THRESHOLDS="INIT_VAL:64,MIN_VAL:1,MAX_VAL:10000"

NOTES

       libmemtier works for applications that do not statically link a malloc implementation.

COPYRIGHT

       Copyright (C) 2021 Intel Corporation. All rights reserved.