Provided by: libmemkind-dev_1.9.0-2_amd64 bug

NAME

       memkind - Heap manager that enables allocations to memory with different properties.
       This  header  expose  STANDARD  and EXPERIMENTAL API. API Standards are described below in
       this man page.

SYNOPSIS

       #include <memkind.h>

       Link with -lmemkind

   EXPERIMENTAL API:
       HEAP MANAGEMENT:
       int memkind_posix_memalign(memkind_t kind, void **memptr, size_t alignment, size_t size);

       KIND MANAGEMENT:
       int   memkind_create_kind(memkind_memtype_t   memtype_flags,   memkind_policy_t    policy,
       memkind_bits_t flags, memkind_t *kind);

   STANDARD API:
       ERROR HANDLING:
       void memkind_error_message(int err, char *msg, size_t size);

       LIBRARY VERSION:
       int memkind_get_version();

       HEAP MANAGEMENT:
       void *memkind_malloc(memkind_t kind, size_t size);
       void *memkind_calloc(memkind_t kind, size_t num, size_t size);
       void *memkind_realloc(memkind_t kind, void *ptr, size_t size);
       void memkind_free(memkind_t kind, void *ptr);
       size_t memkind_malloc_usable_size(memkind_t kind, void *ptr);
       memkind_t memkind_detect_kind(void *ptr);

       KIND CONFIGURATION MANAGEMENT:
       struct memkind_config *memkind_config_new();
       void memkind_config_delete(struct memkind_config *cfg);
       void memkind_config_set_path(struct memkind_config *cfg, const char *pmem_dir);
       void memkind_config_set_size(struct memkind_config *cfg, size_t pmem_size);
       void        memkind_config_set_memory_usage_policy(struct       memkind_config       *cfg,
       memkind_mem_usage_policy policy);

       KIND MANAGEMENT:
       int memkind_create_pmem(const char *dir, size_t max_size, memkind_t *kind);
       int memkind_create_pmem_with_config(struct memkind_config *cfg, memkind_t *kind);
       int memkind_destroy_kind(memkind_t kind);
       int memkind_check_available(memkind_t kind);

       DECORATORS:
       void memkind_malloc_pre(memkind_t *kind, size_t *size);
       void memkind_malloc_post(memkind_t kind, size_t size, void **result);
       void memkind_calloc_pre(memkind_t *kind, size_t *nmemb, size_t *size);
       void memkind_calloc_post(memkind_t kind, size_t nmemb, size_t size, void **result);
       void memkind_posix_memalign_pre(memkind_t *kind, void **memptr, size_t *alignment,  size_t
       *size);
       void  memkind_posix_memalign_post(memkind_t  kind, void **memptr, size_t alignment, size_t
       size, int *err);
       void memkind_realloc_pre(memkind_t *kind, void **ptr, size_t *size);
       void memkind_realloc_post(memkind_t *kind, void *ptr, size_t size, void **result);
       void memkind_free_pre(memkind_t *kind, void **ptr);
       void memkind_free_post(memkind_t kind, void *ptr);

DESCRIPTION

       memkind_error_message() converts an error number err returned by a member of  the  memkind
       interface  to  an error message msg where the maximum size of the message is passed by the
       size parameter.

       HEAP MANAGEMENT:
       The functions described in this section define a heap manager with an interface modeled on
       the  ISO  C  standard API's, except that the user must specify the kind of memory with the
       first argument to each function. See the KINDS section below for a full description of the
       implemented   kinds.    For  file-backed  kind  of  memory  see  memkind_create_pmem()  or
       memkind_create_pmem_with_config().

       memkind_malloc() allocates size bytes of uninitialized memory of the specified kind.   The
       allocated  space  is suitably aligned (after possible pointer coercion) for storage of any
       type of object. If size is 0, then memkind_malloc() returns NULL.

       memkind_calloc() allocates space for num objects each size bytes in length  in  memory  of
       the  specified kind.  The result is identical to calling memkind_malloc() with an argument
       of num * size, with the exception that the allocated memory is explicitly  initialized  to
       zero bytes.  If num or size is 0, then memkind_calloc() returns NULL.

       memkind_realloc() changes the size of the previously allocated memory referenced by ptr to
       size bytes of the specified kind.  The contents of the memory remain unchanged up  to  the
       lesser  of  the  new  and  old sizes. If the new size is larger, the contents of the newly
       allocated portion of the memory are undefined. Upon success, the memory referenced by  ptr
       is freed and a pointer to the newly allocated memory is returned.

       Note:  memkind_realloc()  may  move the memory allocation, resulting in a different return
       value than ptr.

       If ptr is NULL, the memkind_realloc() function behaves identically to memkind_malloc() for
       the  specified  size.   If  size  is  equal to zero, and ptr is not NULL, then the call is
       equivalent to memkind_free(kind, ptr) and NULL is returned. The address ptr, if not  NULL,
       must  have  been  returned  by  a  previous  call  to  memkind_malloc(), memkind_calloc(),
       memkind_realloc() or memkind_posix_memalign() with the same kind as specified to the  call
       to  memkind_realloc().  Otherwise, if memkind_free(kind, ptr) was called before, undefined
       behavior occurs.  In cases where the kind is  unknown  in  the  context  of  the  call  to
       memkind_realloc()  NULL, can be given as the kind specified to memkind_realloc(), but this
       will require a internal look up for correct kind.  Note: The look up for kind could result
       in  serious  performance  penalty,  which can be avoided by specifying a correct kind.  If
       kind is NULL and ptr is NULL, then  memkind_realloc()  returns  NULL  and  sets  errno  to
       EINVAL.

       memkind_posix_memalign()  allocates size bytes of memory of a specified kind such that the
       allocation's base address is an even multiple of alignment, and returns the allocation  in
       the  value pointed to by memptr.  The requested alignment must be a power of 2 at least as
       large as sizeof(void*).  If size is 0, then memkind_posix_memalign()  returns  0,  with  a
       NULL returned in memptr.

       memkind_malloc_usable_size()     function     provides     the     same    semantics    as
       malloc_usable_size(3), but operates on specified kind.
       Note:  In  cases  where  the  kind  is  unknown  in   the   context   of   the   call   to
       memkind_malloc_usable_size()   NULL,   can   be   given   as   the   kind   specified   to
       memkind_malloc_usable_size(), but this could require a internal look up for correct  kind.
       memkind_malloc_usable_size()  is  supported  by  TBB heap manager described in ENVIRONMENT
       section since Intel TBB 2019 Update 4.

       memkind_detect_kind() returns the kind associated with allocated memory referenced by ptr.
       This   pointer   must   have  been  returned  by  a  previous  call  to  memkind_malloc(),
       memkind_calloc(), memkind_realloc() or memkind_posix_memalign().  If  ptr  is  NULL,  then
       memkind_detect_kind()  returns  NULL.   Note:  This  function  has non-trivial performance
       overhead.

       memkind_free() causes the allocated memory referenced by ptr  to  be  made  available  for
       future  allocations.  This  pointer  must  have  been  returned  by  a  previous  call  to
       memkind_malloc(),   memkind_calloc(),   memkind_realloc()   or   memkind_posix_memalign().
       Otherwise,  if  memkind_free(kind,  ptr)  was  already  called  before, undefined behavior
       occurs.  If ptr is NULL, no operation is performed.  In cases where the kind is unknown in
       the  context  of  the  call  to memkind_free() NULL, can be given as the kind specified to
       memkind_free(), but this will require a internal look up for correct kind.  Note: The look
       up  for  kind  could  result  in  serious  performance  penalty,  which  can be avoided by
       specifying a correct kind.

       KIND CONFIGURATION MANAGEMENT:
       The functions described in this section define a way to create,  delete  and  update  kind
       specific  configuration.   Except  of  memkind_config_new(), user must specify the memkind
       configuration with the first argument to each function.  API described here is most useful
       with file-backed kind of memory, e.g.  memkind_create_pmem_with_config() method.

       memkind_config_new() creates the memkind configuration.

       memkind_config_delete()  deletes previously created memkind configuration, which must have
       been returned by a previous call to memkind_config_new().

       memkind_config_set_path() updates the  memkind  pmem_dir  configuration  parameter,  which
       specifies  directory  path,  where file-backed kind of memory will be created.  Note: This
       function does not validate that pmem_dir specifies a valid path.

       memkind_config_set_size() updates the memkind  pmem_size  configuration  parameter,  which
       allows  to  limit  the  file-backed  kind  memory partition.  Note: This function does not
       validate that pmem_size is in valid range.

       memkind_config_set_memory_usage_policy()  updates   the   memkind   policy   configuration
       parameter,  which  allows  to  tune  up memory utilization.  The user should set the value
       based on the characteristics of application that is using  the  library  (e.g.  prioritize
       memory  usage, CPU utilization), for more details about policy see the MEMORY USAGE POLICY
       section below.  Note: This function does not validate that policy is in valid range.

       KIND MANAGEMENT:
       There are built-in kinds that are always available and these are enumerated in  the  KINDS
       section.  The  user  can also create their own kinds of memory. This section describes the
       API's that enable the tracking of the different kinds  of  memory  and  determining  their
       properties.

       memkind_create_pmem()  is  a  convenient  function  used  to  create a file-backed kind of
       memory.  It allocates a temporary file in the given directory dir.  The file is created in
       a  fashion similar to tmpfile(3), so that the file name does not appear when the directory
       is listed and the space is automatically freed when the program terminates.  The  file  is
       truncated to a size of max_size bytes and the resulting space is memory-mapped.
       Note that the actual file system space is not allocated immediately, but only on a call to
       memkind_pmem_mmap() (see memkind_pmem(3)).  This allows to create  a  pmem  memkind  of  a
       pretty  large  size  without  the need to reserve in advance the corresponding file system
       space for the entire heap. If the value of max_size equals 0, pmem memkind is only limited
       by the capacity of the file system mounted under dir argument.  The minimum max_size value
       which allows to limit the size of kind by the library is defined as MEMKIND_PMEM_MIN_SIZE.
       Calling  memkind_create_pmem()  with  a  size  smaller than that and different than 0 will
       return an error.  The maximum allowed size is not limited by  memkind,  but  by  the  file
       system  specified  by  the  dir  argument.   The max_size passed in is the raw size of the
       memory pool and jemalloc will use some of that space for its own metadata.   Returns  zero
       if  the  pmem  memkind is created successfully or an error code from the ERRORS section if
       not.

       memkind_create_pmem_with_config() is a second function used to create a  file-backed  kind
       of  memory.  Function behaves simillar to memkind_create_pmem() but instead of passing dir
       and max_size arguments, it uses config param to specify characteristics of  created  file-
       backed kind of memory (see KIND CONFIGURATION MANAGEMENT section).

       memkind_create_kind() creates kind that allocates memory with specific memory type, memory
       binding policy and flags (see MEMORY FLAGS section).  The memtype_flags (see MEMORY  TYPES
       section) determine memory types to allocate, policy argument is policy for specifying page
       binding to memory types selected by memtype_flags.  Returns zero if the specified kind  is
       created successfully or an error code from the ERRORS section if not.

       memkind_destroy_kind()  destroys  previously  created  kind  object,  which must have been
       returned by a previous call to memkind_create_pmem(), memkind_create_pmem_with_config() or
       memkind_create_kind().   Otherwise,  or  if  memkind_destroy_kind(kind) was already called
       before,  undefined  behavior  occurs.   Note  that,  when  the  kind   was   returned   by
       memkind_create_kind()  all  allocated  memory  must  be  freed  before  kind is destroyed,
       otherwise this will cause memory leak. When the kind was returned by memkind_create_pmem()
       or memkind_create_pmem_with_config() all allocated memory will be freed after kind will be
       destroyed.

       memkind_check_available() returns zero if the specified kind is available or an error code
       from the ERRORS section if it is not.

       MEMKIND_PMEM_MIN_SIZE  The  minimum  size  which  allows  to  limit the file-backed memory
       partition.

       DECORATORS:
       The memkind library enables the user to define decorator  functions  that  can  be  called
       before  and  after each memkind heap management API. The decorators that are called at the
       beginning of the function end are named after that function with _pre appended to the name
       and  those  that  are called at the end of the function are named after that function with
       _post appended to the name. These are weak symbols and if they are  not  present  at  link
       time  they  are  not  called.  The memkind library does not define these symbols which are
       reserved for user definition.  These decorators can be used to track  calls  to  the  heap
       management  interface  or  to  modify  parameters.  The  decorators that are called at the
       beginning of the allocator pass all inputs by reference and the decorators that are called
       at the end of the allocator pass the output by reference. This enables the modification of
       the input and output of each heap management function by the decorators.

       LIBRARY VERSION:
       The memkind library version scheme consist major, minor and  patch  numbers  separated  by
       dot. Combining those numbers, we got the following representation:
       major.minor.patch, where:
            -major   number   is   incremented   whenever   API  is  changed  (loss  of  backward
       compatibility),
            -minor number  is  incremented  whenever  additional  extensions  are  introduced  or
       behavior has been changed,
            -patch number is incremented whenever small bug fixes are added.

       memkind  library  provide  numeric representation of the version by exposing the following
       API:

       memkind_get_version() returns version number  represented  by  a  single  integer  number,
       obtained from the formula:
       major * 1000000 + minor * 1000 + patch

       Note: major < 1 means unstable API.

       API standards:
       -STANDARD API, API is considered as stable
       -NON-STANDARD  API, API is considered as stable, however this is not a standard way to use
       memkind
       -EXPERIMENTAL API, API is considered as unstable and the subject to change

RETURN VALUE

       memkind_calloc(), memkind_malloc()  and  memkind_realloc()  returns  the  pointer  to  the
       allocated  memory  or NULL if the request fails.  memkind_malloc_usable_size() returns the
       number of usable bytes in the block of allocated memory pointed to by ptr, a pointer to  a
       block  of memory allocated by memkind_malloc() or a related function. If ptr is NULL, 0 is
       returned.  memkind_free() and memkind_error_message() do  not  have  return  values.   All
       other  memkind API's return 0 upon success and an error code defined in the ERRORS section
       upon failure.  The memkind library avoids setting errno directly, but calls to  underlying
       libraries and system calls may set errno (e.g.  memkind_create_pmem()).

KINDS

       The available kinds of memory:

       MEMKIND_DEFAULT
              Default allocation using standard memory and default page size.

       MEMKIND_HUGETLB
              Allocate  from  standard  memory  using  huge pages.  Note: This kind requires huge
              pages configuration described in SYSTEM CONFIGURATION section.

       MEMKIND_GBTLB (DEPRECATED)
              Allocate from standard memory using 1GB chunks backed by huge  pages.   Note:  This
              kind requires huge pages configuration described in SYSTEM CONFIGURATION section.

       MEMKIND_INTERLEAVE
              Allocate  pages  interleaved  across  all  NUMA  nodes  with transparent huge pages
              disabled.

       MEMKIND_HBW
              Allocate from the closest high bandwidth memory NUMA node at time of allocation. If
              there  is  not  enough high bandwidth memory to satisfy the request errno is set to
              ENOMEM and the allocated pointer is set to NULL.

       MEMKIND_HBW_ALL
              Same as MEMKIND_HBW except decision regarding closest NUMA node is postponed  until
              the time of first write.

       MEMKIND_HBW_HUGETLB
              Same as MEMKIND_HBW except the allocation is backed by huge pages.  Note: This kind
              requires huge pages configuration described in SYSTEM CONFIGURATION section.

       MEMKIND_HBW_ALL_HUGETLB
              Combination of MEMKIND_HBW_ALL and MEMKIND_HBW_HUGETLB properties.  Note: This kind
              requires huge pages configuration described in SYSTEM CONFIGURATION section.

       MEMKIND_HBW_PREFERRED
              Same  as  MEMKIND_HBW  except  that if there is not enough high bandwidth memory to
              satisfy the request, the allocation will fall back on standard memory.

       MEMKIND_HBW_PREFERRED_HUGETLB
              Same as MEMKIND_HBW_PREFERRED except the allocation is backed by huge pages.  Note:
              This  kind  requires  huge  pages  configuration  described in SYSTEM CONFIGURATION
              section.

       MEMKIND_HBW_GBTLB (DEPRECATED)
              Same as MEMKIND_HBW except the allocation is backed by 1GB chunks  of  huge  pages.
              Note  that  size  can take on any value, but full gigabyte pages will allocated for
              each request, so remainder of the last page will be  wasted.   This  kind  requires
              huge pages configuration described in SYSTEM CONFIGURATION section.

       MEMKIND_HBW_PREFERRED_GBTLB (DEPRECATED)
              Same  as MEMKIND_HBW_GBTLB except that if there is not enough high bandwidth memory
              to satisfy the request, the allocation will fall back on  standard  memory.   Note:
              This  kind  requires  huge  pages  configuration  described in SYSTEM CONFIGURATION
              section.

       MEMKIND_HBW_INTERLEAVE
              Same as  MEMKIND_HBW  except  that  the  pages  that  support  the  allocation  are
              interleaved  across  all  high  bandwidth  nodes  and  transparent  huge  pages are
              disabled.

       MEMKIND_REGULAR
              Allocate from regular memory using the default page  size.  Regular  means  general
              purpose memory from the NUMA nodes containing CPUs.

MEMORY TYPES

       The available types of memory:

       MEMKIND_MEMTYPE_DEFAULT
              Standard memory, the same as process uses.

       MEMKIND_MEMTYPE_HIGH_BANDWIDTH
              High bandwidth memory (HBM). There must be at least two memory types with different
              bandwidth to determine which is the HBM.

MEMORY BINDING POLICY

       The available types of memory binding policy:

       MEMKIND_POLICY_BIND_LOCAL
              Allocate local memory. If there is not enough memory to satisfy the  request  errno
              is set to ENOMEM and the allocated pointer is set to NULL.

       MEMKIND_POLICY_BIND_ALL
              Memory  locality  is  ignored. If there is not enough memory to satisfy the request
              errno is set to ENOMEM and the allocated pointer is set to NULL.

       MEMKIND_POLICY_PREFERRED_LOCAL
              Allocate preferred memory that is local.  If there is not enough  preferred  memory
              to  satisfy  the  request or preferred memory is not available, the allocation will
              fall back on any other memory.

       MEMKIND_POLICY_INTERLEAVE_LOCAL
              Interleave allocation across local memory.  For n memory types the allocation  will
              be interleaved across all of them.

       MEMKIND_POLICY_INTERLEAVE_ALL
              Interleave allocation. Locality is ignored.  For n memory types the allocation will
              be interleaved across all of them.

       MEMKIND_POLICY_MAX_VALUE
              Max policy value.

MEMORY FLAGS

       The available types of memory flags:

       MEMKIND_MASK_PAGE_SIZE_2MB
              Allocation backed by 2MB page size.

MEMORY USAGE POLICY

       The available types of memory usage policy:

       MEMKIND_MEM_USAGE_POLICY_DEFAULT
              Default memory usage policy.

       MEMKIND_MEM_USAGE_POLICY_CONSERVATIVE
              Conservative memory usage policy - prioritize memory usage at cost of  performance.
              Note:  Memory  usage  policies  have  no  effect  for TBB heap manager described in
              ENVIRONMENT section.

ERRORS

       memkind_posix_memalign()
              returns the one of the POSIX standard error codes EINVAL or ENOMEM  as  defined  in
              <errno.h>  if  an  error  occurs  (these  have  positive values).  If the alignment
              parameter is not a power of two or is not a multiple of sizeof(void*), then  EINVAL
              is  returned. If there is insufficient memory to satisfy the request then ENOMEM is
              returned.

       All functions other than memkind_posix_memalign() which have an integer return type return
       one of the negative error codes as defined in <memkind.h> and described below.

       MEMKIND_ERROR_UNAVAILABLE
              Requested memory kind is not available

       MEMKIND_ERROR_MBIND
              Call to mbind(2) failed

       MEMKIND_ERROR_MMAP
              Call to mmap(2) failed

       MEMKIND_ERROR_MALLOC
              Call to jemalloc's malloc() failed

       MEMKIND_ERROR_ENVIRON
              Error parsing environment variable MEMKIND_*

       MEMKIND_ERROR_INVALID
              Invalid input arguments to memkind routine

       MEMKIND_ERROR_TOOMANY
              Error trying to initialize more than maximum MEMKIND_MAX_KIND number of kinds

       MEMKIND_ERROR_BADOPS
              Error memkind operation structure is missing or invalid

       MEMKIND_ERROR_HUGETLB
              Unable to allocate huge pages

       MEMKIND_ERROR_MEMTYPE_NOT_AVAILABLE
              Error requested memory type is not available

       MEMKIND_ERROR_OPERATION_FAILED
              Error memkind operation failed

       MEMKIND_ERROR_ARENAS_CREATE
              Call to jemalloc's arenas.create() failed

       MEMKIND_ERROR_RUNTIME
              Unspecified run-time error

FILES

       /usr/bin/memkind-hbw-nodes
              Prints a comma separated list of high bandwidth nodes.

ENVIRONMENT

       MEMKIND_HBW_NODES
              This  environment variable is a comma separated list of NUMA nodes that are treated
              as high bandwidth. Uses the libnuma routine numa_parse_nodestring() for parsing, so
              the  syntax  described in the numa(3) man page for this routine applies: e.g. 1-3,5
              is a valid setting.

       MEMKIND_ARENA_NUM_PER_KIND
              This environment variable allows leveraging internal mechanism of the  library  for
              setting  number of arenas per kind. Value should be a positive integer (not greater
              than INT_MAX defined in <limits.h>).  The user should set the value  based  on  the
              characteristics  of application that is using the library. Higher value can provide
              better performance in extremely multithreaded applications at the  cost  of  memory
              overhead.  See  section  IMPLEMENTATION NOTES of jemalloc(3) for more details about
              arenas.

       MEMKIND_HOG_MEMORY
              Controls behavior of memkind with regards to returning memory  to  underlaying  OS.
              Setting  MEMKIND_HOG_MEMORY  to  1  causes  memkind  to not release memory to OS in
              anticipation of memory reuse soon. This will improve latency of  'free'  operations
              but increase memory usage.

       MEMKIND_DEBUG
              Controls  logging mechanism in memkind. Setting MEMKIND_DEBUG to 1 enables printing
              messages like errors and general information about environment to stderr.

       MEMKIND_HEAP_MANAGER
              Controls heap management behavior in memkind library by switching  to  one  of  the
              available heap managers.
              Values:
                  JEMALLOC - sets the jemalloc heap manager
                  TBB  -  sets  the  Intel  Threading  Building  Blocks heap manager. This option
              requires installed
                  Intel Threading Building Blocks library.

       If the MEMKIND_HEAP_MANAGER is not set then the jemalloc heap  manager  will  be  used  by
       default.

SYSTEM CONFIGURATION

       Interfaces for obtaining 2MB (HUGETLB) need allocated huge pages in the kernel's huge page
       pool.

       HUGETLB (huge pages)
              Current   number   of    "persistent"    huge    pages    can    be    read    from
              /proc/sys/vm/nr_hugepages  file.  Proposed way of setting hugepages is: sudo sysctl
              vm.nr_hugepages=<number_of_hugepages>.   More  information  can  be   found   here:
              ⟨https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt⟩

STATIC LINKING

       When  linking  statically  against  memkind, libmemkind.a should be used together with its
       dependencies libnuma and pthread. Pthread can be linked by adding  /usr/lib64/libpthread.a
       as  a  dependency  (exact  path may vary). Typically libnuma will need to be compiled from
       sources  to  use  it  as  a  static  dependency.   libnuma  can  be  reached  on   GitHub:
       ⟨https://github.com/numactl/numactl⟩

KNOWN ISSUES

       HUGETLB (huge pages)
              There  might  be some overhead in huge pages consumption caused by heap management.
              If your allocation fails because of OOM, please try to allocate  extra  huge  pages
              (e.g. 8 huge pages).

COPYRIGHT

       Copyright (C) 2014 - 2019 Intel Corporation. All rights reserved.

SEE ALSO

       malloc(3),  malloc_usable_size(3),  numa(3), numactl(8), mbind(2), mmap(2), move_pages(2),
       jemalloc(3),  memkind_default(3),  memkind_arena(3),  memkind_hbw(3),  memkind_hugetlb(3),
       memkind_pmem(3)