Provided by: libmemkind-dev_1.14.0-2_amd64 bug

NAME

       hbwmalloc - The high bandwidth memory interface
       Note: hbwmalloc.h functionality is considered as stable API (STANDARD API).

SYNOPSIS

       #include <hbwmalloc.h>

       Link with -lmemkind

       int hbw_check_available(void);
       void* hbw_malloc(size_t size);
       void* hbw_calloc(size_t nmemb, size_t size);
       void* hbw_realloc (void *ptr, size_t size);
       void hbw_free(void *ptr);
       size_t hbw_malloc_usable_size(void *ptr);
       int hbw_posix_memalign(void **memptr, size_t alignment, size_t size);
       int hbw_posix_memalign_psize(void **memptr, size_t alignment, size_t size, hbw_pagesize_t pagesize);
       hbw_policy_t hbw_get_policy(void);
       int hbw_set_policy(hbw_policy_t mode);
       int hbw_verify_memory_region(void *addr, size_t size, int flags);

DESCRIPTION

       hbw_check_available()  returns zero if high bandwidth memory is available or an error code
       described in the ERRORS section if not.

       hbw_malloc() allocates size bytes of uninitialized high bandwidth  memory.  The  allocated
       space  is  suitably  aligned  (after possible pointer coercion) for storage of any type of
       object. If size is zero then hbw_malloc() returns NULL.

       hbw_calloc() allocates space for nmemb objects in high bandwidth memory, each  size  bytes
       in  length.  The  result  is identical to calling hbw_malloc() with an argument of nmemb *
       size, with the exception that the allocated  memory  is  explicitly  initialized  to  zero
       bytes.  If nmemb or size is 0, then hbw_calloc() returns NULL.

       hbw_realloc()  changes  the  size  of  the  previously  allocated  high  bandwidth  memory
       referenced by ptr to size bytes. The contents of the memory remain  unchanged  up  to  the
       lesser  of  the  new  and  old sizes. If the new size is larger, the contents of the newly
       allocated portion of the memory are undefined. Upon success, the memory referenced by  ptr
       is freed and a pointer to the newly allocated high bandwidth memory is returned.

       Note:  hbw_realloc() may move the memory allocation, resulting in a different return value
       than ptr.

       If ptr is NULL, the hbw_realloc() function behaves identically  to  hbw_malloc()  for  the
       specified  size.   If  size  is  equal  to  zero,  and  ptr  is not NULL, then the call is
       equivalent to hbw_free(ptr) and NULL is returned.  The  address  ptr,  if  not  NULL,  was
       returned   by   a   previous   call   to   hbw_malloc(),  hbw_calloc(),  hbw_realloc()  or
       hbw_posix_memalign().   Otherwise,  or  if  hbw_free(ptr)  was  called  before,  undefined
       behavior occurs.

       Note: hbw_realloc() cannot be used with a pointer returned by hbw_posix_memalign_psize().

       hbw_free()  causes  the allocated memory referenced by ptr to be made available for future
       allocations. If ptr is NULL, no action occurs.  The address ptr, if not  NULL,  must  have
       been   returned   by   a  previous  call  to  hbw_malloc(),  hbw_calloc(),  hbw_realloc(),
       hbw_posix_memalign()  or  hbw_posix_memalign_psize().   Otherwise,  if  hbw_free(ptr)  was
       called before, undefined behavior occurs.

       hbw_malloc_usable_size()  returns  the  number  of usable bytes in the block pointed to by
       ptr,  a  pointer  to  a  block  of  memory  allocated   by   hbw_malloc(),   hbw_calloc(),
       hbw_realloc(), hbw_posix_memalign(), or hbw_posix_memalign_psize().

       hbw_posix_memalign()  allocates  size  bytes  of  high  bandwidth  memory  such  that  the
       allocation's base address is an even multiple of alignment, and returns the allocation  in
       the  value pointed to by memptr.  The requested alignment must be a power of 2 at least as
       large as sizeof(void*).  If size is 0, then hbw_posix_memalign() returns 0,  with  a  NULL
       returned in memptr.

       hbw_posix_memalign_psize()  allocates  size  bytes  of high bandwidth memory such that the
       allocation's base address is an even multiple of alignment, and returns the allocation  in
       the  value pointed to by memptr.  The requested alignment must be a power of 2 at least as
       large as sizeof(void*).  The memory will  be  allocated  using  pages  determined  by  the
       pagesize variable which may be one of the following enumerated values:

       HBW_PAGESIZE_4KB
              The  four  kilobyte page size option. Note that with transparent huge pages enabled
              these allocations may be promoted by the operating system to two megabyte pages.

       HBW_PAGESIZE_2MB
              The two megabyte page size option.   Note:  This  page  size  requires  huge  pages
              configuration described in SYSTEM CONFIGURATION section.

       HBW_PAGESIZE_1GB (DEPRECATED)
              This option allows the user to specify arbitrary sizes backed by 1GB chunks of huge
              pages. Huge pages are allocated even if the size is not a  modulo  of  1GB.   Note:
              This  page size requires huge pages configuration described in SYSTEM CONFIGURATION
              section.

       HBW_PAGESIZE_1GB_STRICT (DEPRECATED)
              The total size of the allocation must be  a  multiple  of  1GB  with  this  option,
              otherwise  the  allocation  will  fail.   Note:  This page size requires huge pages
              configuration described in SYSTEM CONFIGURATION section.

       Note: HBW_PAGESIZE_2MB,  HBW_PAGESIZE_1GB  and  HBW_PAGESIZE_1GB_STRICT  options  are  not
       supported with HBW_POLICY_INTERLEAVE policy.

       hbw_get_policy()  returns  the  current  fallback  policy when insufficient high bandwidth
       memory is available.

       hbw_set_policy() sets the current fallback policy. The policy can be modified only once in
       the   lifetime   of   an   application  and  before  calling  hbw_malloc(),  hbw_calloc(),
       hbw_realloc(), hbw_posix_memalign(), or hbw_posix_memalign_psize() function.
       Note: If the policy is not set, than HBW_POLICY_PREFERRED will be used by default.

       HBW_POLICY_BIND
              If insufficient high bandwidth memory from the nearest NUMA node  is  available  to
              satisfy a request, the allocated pointer is set to NULL and errno is set to ENOMEM.
              If insufficient high bandwidth memory pages are available at fault time the Out  Of
              Memory (OOM) Killer is triggered.  Note that pages are faulted exclusively from the
              high bandwidth NUMA node nearest at time of allocation, not at time of fault.

       HBW_POLICY_BIND_ALL
              If insufficient high bandwidth memory  is  available  to  satisfy  a  request,  the
              allocated  pointer is set to NULL and errno is set to ENOMEM.  If insufficient high
              bandwidth memory pages are available at fault time the Out Of Memory  (OOM)  Killer
              is  triggered.   Note  that  pages  are faulted from the high bandwidth NUMA nodes.
              Nearest NUMA node is selected at time of page fault.

       HBW_POLICY_PREFERRED
              If insufficient memory is available from the high bandwidth NUMA  node  closest  at
              allocation  time,  fall  back  to  standard memory (default) with the smallest NUMA
              distance.

       HBW_POLICY_INTERLEAVE
              Interleave faulted pages from across all high bandwidth NUMA nodes  using  standard
              size pages (the Transparent Huge Page feature is disabled).

       hbw_verify_memory_region()  verifies  if  memory  region  fully  falls into high bandwidth
       memory. Returns 0 if memory address range from addr to addr + size is  allocated  in  high
       bandwidth  memory,  -1  if  any fragment of memory was not backed by high bandwidth memory
       (e.g. when memory is not initialized) or one of error codes described in ERRORS section.

       Using this function in production code may result in serious performance penalty.

       The Flags argument may include optional flags that modify function behavior:

       HBW_TOUCH_PAGES
              Before checking pages, function will touch first byte of all pages in address range
              starting  from  addr  to  addr  +  size  by  read and write (so the content will be
              overwritten by the same data as it was read).  Using this option may trigger Out Of
              Memory Killer.

RETURN VALUE

       hbw_get_policy()  returns  HBW_POLICY_BIND,  HBW_POLICY_BIND_ALL,  HBW_POLICY_PREFERRED or
       HBW_POLICY_INTERLEAVE which represents the current high bandwidth policy.   hbw_free()  do
       not  have return value.  hbw_malloc() hbw_calloc() and hbw_realloc() return the pointer to
       the  allocated  memory,   or   NULL   if   the   request   fails.    hbw_posix_memalign(),
       hbw_posix_memalign_psize() and hbw_set_policy() return zero on success and return an error
       code as described in the ERRORS section below on failure.

ERRORS

       Error codes described here are the POSIX standard error codes as defined in
              <errno.h>

       hbw_check_available()
              returns ENODEV if high-bandwidth memory is unavailable.

       hbw_posix_memalign() and hbw_posix_memalign_psize()
              If the alignment parameter is not a  power  of  two,  or  was  not  a  multiple  of
              sizeof(void*),  then EINVAL is returned.  If the policy and pagesize combination is
              unsupported then EINVAL is returned.  If there was insufficient memory  to  satisfy
              the request then ENOMEM is returned.

       hbw_set_policy()
              returns  EPERM  if  hbw_set_policy()  was  called more than once, or EINVAL if mode
              argument was neither HBW_POLICY_PREFERRED, HBW_POLICY_BIND, HBW_POLICY_BIND_ALL nor
              HBW_POLICY_INTERLEAVE.

       hbw_verify_memory_region()
              returns  EINVAL  if  addr is NULL, size equals 0 or flags contained unsupported bit
              set. If memory pointed by addr could not be verified then EFAULT is returned.

NOTES

       The <hbwmalloc.h> file defines the external functions and enumerations for  the  hbwmalloc
       library.  These  interfaces  define a heap manager that targets high bandwidth memory numa
       nodes.

FILES

       /usr/bin/memkind-hbw-nodes
              Prints a comma-separated list of high bandwidth nodes.

ENVIRONMENT

       MEMKIND_HBW_NODES
              This environment variable is a comma-separated list of NUMA nodes that are  treated
              as high bandwidth. Uses the libnuma routine numa_parse_nodestring() for parsing, so
              the syntax described in the numa(3) man page for this routine applies for  example:
              1-3,5 is a valid setting.

       MEMKIND_ARENA_NUM_PER_KIND
              This  environment  variable allows leveraging internal mechanism of the library for
              setting number of arenas per kind. Value should be a positive integer (not  greater
              than  INT_MAX  defined  in <limits.h>).  The user should set the value based on the
              characteristics of application that is using the library. Higher value can  provide
              better  performance  in  extremely multithreaded applications at the cost of memory
              overhead. See section IMPLEMENTATION NOTES of jemalloc(3) for  more  details  about
              arenas.

       MEMKIND_HEAP_MANAGER
              Controls  heap  management  behavior  in memkind library by switching to one of the
              available heap managers.
              Values:
                  JEMALLOC - sets the jemalloc heap manager
                  TBB - sets the Intel  Threading  Building  Blocks  heap  manager.  This  option
              requires installed
                  Intel Threading Building Blocks library.

       Note:  If  the MEMKIND_HEAP_MANAGER is not set then the jemalloc heap manager will be used
       by default.

SYSTEM CONFIGURATION

       Interfaces for obtaining 2MB (HUGETLB) memory need allocated huge pages  in  the  kernel's
       huge page pool.

       HUGETLB (huge pages)
              Current    number    of    "persistent"    huge    pages    can    be   read   from
              /proc/sys/vm/nr_hugepages file.  Proposed way of setting hugepages is: sudo  sysctl
              vm.nr_hugepages=<number_of_hugepages>.    More   information  can  be  found  here:
              ⟨https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt⟩

KNOWN ISSUES

       HUGETLB (huge pages)
              There might be some overhead in huge pages consumption caused by  heap  management.
              If  your  allocation  fails because of OOM, please try to allocate extra huge pages
              (e.g. 8 huge pages).

COPYRIGHT

       Copyright (C) 2014 - 2020 Intel Corporation. All rights reserved.

SEE ALSO

       malloc(3), numa(3), numactl(8), mbind(2), mmap(2), move_pages(2), jemalloc(3), memkind(3)